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ABSTRACT 


Naval  Aviation  has  been  challenged  to  cut  its  1996  human  factors  related  Class  A 
flight  mishap  rate  in  half  by  the  year  2000.  Investigations  show  that  human  caused  flight 
mishaps  have  not  declined  as  rapidly  as  mechanical  ones.  From  fiscal  year  1990  through 
1997,  maintenance  was  a  causal  factor  in  17  percent  of  Class  A  flight  mishaps.  Presently, 
there  is  an  ongoing  effort  to  identify  factors  contributing  to  human  error  in  aviation 
maintenance.  One  major  component  is  the  development  of  an  instrument  to  assess  safety 
climate  and  posture  in  maintenance  operations.  This  thesis  is  the  climate  safety 
assessment  portion  of  this  effort.  It  utilizes  and  adapts  an  existing  Model  of 
Organizational  Safety  Effectiveness  (MOSE)  to  achieve  an  understanding  of  the  possible 
influences  of  organizational  factors  on  aviation  maintenance.  This  thesis  develops  and 
administers  a  prototype  Maintenance  Climate  Assessment  Survey  (MCAS)  that  provides 
a  tool  for  assessing  safety  in  maintenance  operations.  The  study  has  268  participants 
from  three  Reserve  squadrons  that  represent  the  spectrum  of  aviation  communities.  The 
prototype  MCAS  is  comprised  of  67  questions  developed  from  155  candidate  questions. 
Each  question  uses  a  Likert  type  rating  scale,  which  allows  participants  to  express 
opinions  for  each  item  presented.  Cluster  and  Factor  analysis  is  used  to  identify 
redundancies  between  items  and  how  items  clustered  according  to  the  MOSE 
components.  The  product  of  this  study  is  a  finalized  MCAS  with  35  questions  that  can  be 
used  by  the  Squadron  command  and  Aviation  Safety  Officer  to  assess  their  unit’s  safety 
posture  in  conducting  scheduled/unscheduled  maintenance  operations. 
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EXECUTIVE  SUMMARY 


Naval  Aviation  has  been  challenged  to  cut  its  1996  human  factors  related  Class  A 
flight  mishap  rate  in  half  by  the  year  2000.  Numerous  engineering  and  programmatic 
intervention  strategies  have  been  put  in  place  to  reduce  aviation  mishaps,  and 
subsequently,  the  annual  flight  mishap  rate.  Improved  aircraft  reliability,  advanced 
cockpit  technologies,  upgrades  to  avionics,  and  new  acquisitions  have  lead  to 
improvements  that  have  helped  to  lower  the  mishap  rate.  However,  mishaps  are  still 
occurring.  Investigations  show  that  while  the  mechanical-caused  flight  mishap  rate  has 
fallen  significantly  to  an  all  time  low,  the  human-caused  flight  mishap  rate  has  not 
declined  as  rapidly  and  has  possibly  increased  in  the  last  10  years  (Naval  Safety  Center, 
1997). 

On  the  heels  of  the  29  January  1996  Nashville,  Tennessee,  F-14  fighter  jet  mishap 
that  killed  the  aircrew  and  several  local  residents.  Vice  Admiral  Bennitt,  Commander 
Naval  Air  Force,  U.S.  Pacific  Fleet,  created  a  Human  Factors  Quality  Management  Board 
(HFQMB)  to  improve  aviation  safety  by  preventing  human  error.  The  HFQMB  was 
chartered  to  examine  factors  contributing  to  human  error,  especially  aircrew  error. 

In  1997,  the  Navy  and  Marine  Corps  had  an  impressive  year;  their  combined  class 
A  flight  mishap  rate  was  the  second  lowest  ever  recorded  and  it  was  the  Navy’s  best  ever 
recorded  (NSC,  1998).  Given  this  success,  a  shift  to  improve  maintenance  was  initiated. 
Of  all  the  identified  causal  factors  that  contributed  to  Class  A  flight  mishaps  from  fiscal 
year  1990  through  1997,  17  percent  involved  maintenance.  “Maintenance”  causal  factors 
are  primarily  due  to  human  errors  by  maintenance  personnel  and/or  maintenance 
supervisors.  The  maintenance  thrust  of  the  HFQMB  adapted  the  same  approach  of 
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benchmarking,  mishap  data  analysis,  and  climate  safety  assessment.  This  thesis  is  the 
climate  safety  assessment  portion  of  an  ongoing  study  of  Naval  Aviation  maintenance 
operations. 

Organizational  culture  has  an  effect  on  the  entire  organization.  This  is  especially 
true  in  high-risk  organizations  where  potential  outcomes  of  a  system  failure  can  be 
catastrophic  and  costly.  Naval  Aviation  is  recognized  as  a  high-risk  organization,  which 
can  experience  devastation  if  something  goes  awry.  This  also  holds  true  for  aviation 
maintenance  as  well  as  operations.  This  study  involves  the  analysis  of  data  from  a 
prototype  survey  that  is  based  on  an  existing  model  of  high-risk  organizations.  The 
purpose  of  this  study  is  to  develop,  administer,  and  validate  a  Maintenance  Climate 
Assessment  Survey  (MCAS)  to  assess  the  effectiveness  of  Naval  Aviation  maintenance 
operations  in  the  management  of  risk.  It  utilizes  and  adapts  an  existing  Model  of 
Organizational  Safety  Effectiveness  (MOSE).  This  study  analyzes  the  MCAS  data;  the 
results  are  intended  to  improve  our  understanding  of  the  possible  influences  of  human 
factors  in  aviation  maintenance  related  mishaps,  and  to  provide  a  tool  that  can  be  used  by 
the  Squadron  command  and  Aviation  Safety  Officer  to  assess  their  unit’s  safety  posture 
in  conducting  scheduled  /  unscheduled  maintenance  operations. 

The  study  includes  a  total  of  268  participants,  of  whom  212  are  maintainers,  from 
three  Reserve  squadrons  that  represent  a  variety  of  Naval  Aviation  communities.  The 
prototype  MCAS  comprised  67  questions  developed  from  155  candidate  questions.  Each 
question  contained  five  point  Likert  scales  (strongly  disagree,  disagree,  neutral,  agree, 
and  strongly  agree)  which  allowed  participants  to  rate  items  according  to  their  views. 
The  questions  fit  into  six  components  of  the  MOSE:  Process  Auditing,  Reward  System 
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and  Safety  Culture,  Quality,  Risk  Management,  Command  and  Control,  and 
Communication  /  Relationships.  The  data  analysis  entails  descriptive  statistics,  cluster 
analysis  and  factor  analysis  for  objective  identifications  of  redundancies  among  items  and 
how  items  clustered  according  to  the  MOSE  components.  Redundant  questions  are 
eliminated  and  questions  with  common  intent  are  combined.  Some  questions  remain  the 
same.  Thorough  analysis  of  the  questions  produce  a  finalized  MCAS  with  a  total  of  35 
questions  that  can  be  used  for  individual  squadron  self-assessment  of  safety  posture  in 
scheduled  /  unscheduled  maintenance  operations. 
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I. 


INTRODUCTION 


Naval  Aviation  is  an  inherently  complicated  system,  from  flying  the  platform  into  a 
hostile  environment  to  landing  it  on  an  aircraft  carrier.  Complex  systems  have  many 
forces  acting  on  them,  some  synergistically,  some  separately,  all  having  either  positive  or 
negative  effects  (Peirow,  1986).  Such  forces  and  the  effects  they  have  on  the  system  are 
difficult  to  grasp  and  understand  (Bond,  Byran,  Rigney,  &  Warren,  1975).  These  forces 
can  lead  to  pilot  errors,  cause  failures  in  the  platform  or  create  a  range  of  causal  factors 
that  can  lead  to  the  loss  of  a  platform,  or  worse,  lives,  thus  having  a  direct  effect  on 
combat  readiness.  Warfighting  readiness  and  victory  in  combat  can  be  achieved  by 
keeping  the  platforms  functional  and  the  pilots  alive  (Nutwell  &  Sherman,  1997). 

A.  BACKGROUND 

The  Naval  Aviation  Safety  Program  (Chief  of  Naval  Operations  Instruction 
3750.6Q,  1989)  was  established  to  deal  with  aviation  mishaps  and  any  related  causal 
factors.  The  purpose  of  this  program  is  to  preserve  human  and  material  resources  while 
its  objective  is  to  eliminate  hazards,  the  causes  of  damage  and  injury.  In  essence,  the 
program  was  set  up  to  determine  all  hazards  that  can  become  causal  factors  leading  to 
aviation  mishaps. 

Numerous  efforts  have  been  employed  in  attempts  to  lower  the  Class  A  flight 
mishap  rate  (Figure  1).  These  efforts  resulted  in  the  creation  of  several  institutions  and 
programs,  including  the  Naval  Aviation  Safety  Center  (now  Naval  Safety  Center  -  NSC), 
the  Naval  Aviation  Maintenance  Program  (NAMP),  the  Fleet  Replacement  Squadron 
(FRS)  concept,  the  Naval  Air  Training  &  Operating  Procedures  &  Standards  (NATOPS), 
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and  the  Squadron  Safety  Program  (SSP).  Additional  efforts  include  Aircrew  Coordination 
Training  (ACT),  Human  Factors  Councils/Boards  (HFC/B),  and  Operational  Risk 
Management  (ORM). 


77fi  airrraft  aircraft 
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Figure  1.  FY  50-96  Flight  Mishap  Rates  and  Intervention  Strategies. 

From  “Naval  Safety  Center  Brief,”  by  Admiral  F.  Dirren,  April  1997. 

All  of  the  above  efforts  are  intervention  strategies  put  in  place  to  reduce  aviation 
mishaps  and  the  annual  flight  mishap  rate.  The  first  five  programs  mentioned  are  design 
and  programmatic  in  type.  The  last  three  break  from  an  engineering  and  management 
focus  to  concentrate  more  on  human  factors  issues.  Improved  aircraft  reliability, 
advanced  cockpit  technologies,  upgrades  to  avionics,  and  new  acquisitions  have  lead  to 
improvements  that  lower  the  mishap  rate.  However,  mishaps  are  still  occurring. ,  Since 
1990,  approximately  4  of  every  5  Department  of  the  Navy  (DON)  Class  A  flight  mishaps 
(those  involving  aircraft  damage  in  excess  of  one  million  dollars  or  fatal  injury)  involve 
human  factors  (Department  of  the  Navy  (DON),  1997).  For  example,  in  1997,  of  the  27 
aircraft  destroyed,  23  involved  human  factors. 
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Naval  Aviation  has  been  challenged  to  cut  its  1996  human  factor  related  Class  A 
flight  mishap  rate  in  half  by  the  year  2000.  To  do  this,  it  must  systematically  and 
continuously  improve  all  of  the  processes,  programs,  and  systems  affecting  the  safety  of 
Naval  Aviation  operations  (Nutwell  &  Sherman,  1997).  The  Naval  Aviation  Class  A 
flight  mishap  rate  has  been  significantly  reduced  over  the  past  few  decades.  However,  it 
appears  to  have  reached  a  plateau  over  the  past  decade  as  indicated  in  Figure  2. 

4i 


88  89  90  91  92  93  94  95  96  97 

Year 

Figure  2.  Naval  Aviation  Class  A  Flight  Mishap  Trend  FY  88-97. 


Rate  is  Defined  as  the  Number  of  Class  A,  B,  &  C  Flight  Mishaps  Per  100,000  Flight  Hours. 

Though  the  Navy  and  Marine  Corps  flight  mishap  rate  has  generally  decreased 
over  the  past  two  decades,  the  rate  of  decline  has  slowed.  Investigations  show  that  while 
the  mechanical-caused  flight  mishap  rate  has  fallen  significantly  to  an  all  time  low,  the 
human-caused  flight  mishap  rate  has  not  declined  as  rapidly  and  is  possibly  leveling  off 
or  even  increasing  in  recent  years  (NSC,  1997).  Some  conclude  that  this  split  is  due  to 
the  complexity  of  the  aircraft  and  current  pressures  of  operations  tempo,  personnel 
downsizing,  etc  (Schmidt,  1998).  Figure  3  shows  this  relationship. 
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Figure  3.  Mechanical/Catastrophic  Failures  vs  Human  Errors  Over  All  Mishaps. 

From  “Naval  Safety  Center  Brief,”  by  Admiral  F.  Dirren,  April  1997. 

Figure  4  shows  the  percentage  of  determined  causal  factors  that  contributed  to 
Class  A  flight  mishaps  from  FY90  -  FY97.  Supervisory  factors  were  present  in  62 
percent  of  these  flight  mishaps.  Examples  of  supervisory  factors  include  inadequate 
command  supervision  of  flight  operations  or  safety,  failures  to  correctly  assess  mission 
risks,  and  the  inappropriate  handling  of  a  known  high-risk  situation.  The  aircrew  was 
found  to  be  a  causal  factor  in  56  percent  of  the  mishaps.  Material  factors,  defined  as 
material  failure  of  the  component,  were  present  in  39  percent  of  the  mishaps. 
Maintenance  was  found  to  be  a  causal  factor  in  17  percent  of  mishaps.  This 
“maintenance”  causal  factor  was  primarily  due  to  human  errors  by  maintenance  personnel 
or  maintenance  supervisors.  Facilities  factors,  relating  to  human  errors  in  conjunction 
with  airfield  operations,  contributed  to  five  percent  of  the  mishaps. 
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Figure  4.  Navy  and  Marine  Corps  Fiscal  Year  90-96  Class  A  Flight  Mishap  Causal  Factors. 

From  “Naval  Safety  Center  Brief,”  by  Admiral  F.  Dirren,  April  1997. 

On  the  heels  of  the  29  January  1996  Nashville,  Tennessee,  F-14  fighter  jet  mishap 
that  killed  the  aircrew  and  several  local  residents.  Vice  Admiral  Bennitt,  Commander 
Naval  Air  Force,  U.S.  Pacific  Fleet,  created  a  Human  Factors  Quality  Management  Board 
(HFQMB)  to  improve  aviation  safety  by  preventing  human  error.  The  charter  of  the 
HFQMB  is  to  analyze  and  improve  each  of  the  processes,  programs,  and  systems  that 
impact  human  performance  in  aviation  with  the  purpose  of  dramatically  reducing  the 
annual  flight  mishap  rate  (DON,  1997).  In  other  words,  its  main  objective  is  to  analyze 
human  factors  involvement  in  past  Naval  Aviation  mishaps  and  in  present  Naval  Aviation 
operations.  Preventing  a  mishap  contributes  to  readiness  and  mission  success  by  keeping 
people  and  airplanes  available  through  controlling  safety-related  hazards.  Most  of  the 
HFQMB  initiatives  directly  contribute  to  mission  readiness,  as  well  as  indirectly,  through 
mishap  prevention  (Nutwell  &  Sherman,  1997). 
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Three  processes  define  the  HFQMB  methodology:  Benchmarking,  Mishap  Data- 
Analysis  (MDA),  and  Command  Safety  Assessment  (CSA).  These  lead  to  brainstorming 
and  potential  intervention  strategies,  which  in  turn  result  in  recommendations  presented 
to  the  Navy’s  Air  Board  of  Flag  officers  (Figure  5).  Benchmarking  occurred  with  visits 
to  commercial  airlines,  private  industry,  sister  services,  NASA,  the  FAA,  NATO  partners, 
and  academia.  The  objective  was  to  examine  the  culture,  training,  safety,  and  leadership 
of  each  organization;  to  identify  and  learn  about  successful  safety  practices  in  order  to 
adopt  best  practices,  identify  lessons  learned,  and  avoid  “re-inventing  the  wheel.” 


Figure  5.  HFQMB  Methodology. 

From  “AIRPAC  Brief,”  by  CDR  J.  Schmidt,  March  1998. 

The  MDA  categorizes  the  human  error  elements  of  a  mishap  into  three  groups: 
unsafe  supervisory  conditions,  unsafe  aircrew  conditions,  and  unsafe  aircrew  acts.  The 
first  two  are  seen  as  latent  conditions  and  the  last  as  active  failures.  Unsafe  supervisory 
conditions  include  planned  inappropriate  operations,  failure  to  correct  known  problems, 
supervisory  violations,  and  inadequate  supervision.  Unsafe  aircrew  conditions  pertain  to 
aeromedical,  personnel  readiness  and  cockpit  resource  management  (CRM)  problems. 
Unsafe  aircrew  acts  pertain  to  slips,  lapses,  mistakes,  routine  violations,  and  exceptional 
acts  of  the  aircrew.  Class  A  flight  mishap  data  analysis  from  fiscal  year  1990  through 
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1996  showed  the  following.  Supervisory  failure  is  a  causal  factor  in  62  percent  of  Class 
A  flight  mishaps  during  fiscal  year  1990  through  1996.  Fifty  one  percent  of  the 
supervisory  failures  are  labeled  as  unsafe  supervisory  conditions.  The  unsafe  aircrew 
events  contained:  28  percent  of  the  flight  mishaps  as  aeromedical,  48  percent  as  CRM 
and  three  percent  as  readiness.  The  unsafe  aircrew  act  event  contained  87  percent  as 
accidental  and  40  percent  as  violations.  (Nutwell  &  Sherman,  1997).  Notably,  in  a  report 
issued  in  1997  by  the  DON,  forty  percent  of  the  volations  were  conscious  aircrew 
breeches  of  procedures.  Standard  Operating  Procedures  (SOP)  and  NATOPS. 

Ciavarelli’s  and  Figlock’s  (1997)  CSA  study  was  conducted  to  assess  a 
command’s  safety  climate  from  an  aircrew  perspective  and  the  perceived  effectiveness  of 
its  safety  related  programs.  Its  objective  was  to  analyze  the  effectiveness  of  managing 
risks  in  flight  operations.  The  study,  based  on  a  model  of  high  reliability  organizations, 
shows  significant  room  for  improvement  in  risk  management,  human  factors  evaluation, 
and  aircrew  coordination  training.  The  study’s  survey  develops  a  tool  that  provides 
commanding  officers  and  aviation  safety  officers  (ASO)  a  capability  for  self  assessment 
of  the  effectiveness  of  their  commands  safe  operations  (Nutwell  &  Sherman,  1997). 

The  HFQMB  formed  ten  Process  Action  Teams  (PAT)  for  areas  that  require 
intervention:  Leadership,  Policy,  Organizational  Effectiveness,  Safety  Information 
Management,  Training  and  Qualifications,  Standard  Operating  Procedures,  Mishap 
Investigations,  Operational  Risk  Management,  Human  Factors  Evaluation  and 
Aircraft/Aircrew  Systems.  The  PAT  were  tasked  to  assess  each  area  using  the 
information  taken  from  benchmarking,  the  MDA,  and  CSA,  to  identify  area 
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improvements,  and  recommend  actions  resulting  in  an  awareness  of  human  factors,  causal 
factors  of  mishaps  and,  ultimately,  to  achieve  a  mishap  rate  reduction. 

To  date,  there  has  been  little  emphasis  on  the  potential  contribution  of  human 
factors  on  maintenance  operations.  The  HFQMB’s  first  thrust  was  aimed  at  supervisory 
(62  percent)  and  aircrew  (56  percent)  causal  factors  due  to  the  large  percentage  of 
mishaps  involved,  whereas  discussed  earlier,  maintenance  was  found  in  17  percent  of  the 
mishaps.  In  1997,  the  Navy  and  Marine  Corps  had  an  impressive  year;  its  class  A  flight 
mishap  rate  was  the  second  lowest  ever  recorded  and  it  was  the  Navy’s  best  ever  recorded 
(NSC,  1998).  Success  in  the  intervention  of  these  two  areas  has  led  to  interest  in 
developing  efforts  directed  at  the  maintenance  causal  factors. 

The  same  three-pronged  approach  was  adopted  by  the  HFQMB  for  taking  an  in- 
depth  look  into  Naval  Aviation  maintenance  operations  for  the  identification  of  hazards 
and  the  development  of  intervention  possibilities  (Figure  5)  (Schmidt,  1998). 
Benchmarking  is  currently  underway  and  mishap  data  analysis  is  now  completed.  The 
benchmarking  efforts  have  determined  that  risk  management  contributes  to  enhanced 
safety  in  terms  of  maintenance  error  prevention,  lower  mishaps,  and  reduced  personal 
injuries.  Organizational  culture  and  climate,  facilitation  of  crew  coordination  and  team 
performance,  effective  training  and  certificate  programs,  and  emphasis  on  policy  and 
procedure  adherence  are  all  areas  where  risk  management  contributes  to  enhanced  safety. 
These  findings  reflect  some  of  the  problem  areas  found  in  an  analysis  of  Class  A  flight 
mishaps  that  involved  maintenance  depicted  issues  related  to  supervisory,  crew 
coordination,  training/qualifications,  and  adherence  to  policy  and  procedures.  The 
results  from  benchmarking  and  mishap  data  analysis  in  conjunction  with  the  previous 
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CSA  questionnaire  were  used  to  develop  a  candidate  list  of  questions  for  a  maintenance 
questionnaire. 

B.  PURPOSE 

The  purpose  of  this  study  is  to  develop,  administer,  and  validate  a  questionnaire 
survey  to  assess  the  effectiveness  of  Naval  Aviation  maintenance  operations  in  the 
management  of  risk.  This  study  analyzes  survey  data  and  builds  on  a  Model  of 
Organizational  Safety  Effectiveness  (MOSE)  developed  by  Dr.  Ciavarelli  and  LtCol 
Figlock  (1996)  of  the  School  of  Aviation  Safety,  Naval  Postgraduate  School,  Monterey, 
CA.  The  results  will  improve  our  understanding  of  the  possible  influences  of  human 
factors  in  aviation  maintenance  related  mishaps,  and  will  provide  a  tool  for  assessing 
safety  in  maintenance  operations.  The  intent  is  to  identify  potential  interventions  in  an 
effort  to  lower  human  error  in  maintenance-related  mishaps  and  the  overall  mishap  rate. 

C.  PROBLEM  STATEMENT 

Given  the  sustained  presence  of  human  error  in  aviation  maintenance,  it  is 
essential  to  develop  a  valid  Maintenance  Climate  Assessment  Survey  targeting  the 
maintainer’s  perspective  on  safety  and  flight  practices.  Given  this  need,  the  existing  CSA 
questionnaire  based  on  the  present  MOSE  must  be  adapted  to  address  maintenance  issues. 
It  entails  possible  expansion  and  necessary  modification  of  the  existing  questionnaire, 
which  is  geared  towards  aircrew,  and  the  development  of  maintenance  specific  items. 
This  thesis  explores  the  following  questions: 

1.  Can  the  present  MOSE  be  adapted  to  capture  the  maintainer’s  perspective? 

2.  Do  the  close-ended  questions  (questions  with  fixed  responses)  in  the  MCAS 
conform  to  the  existing  MOSE  components? 
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3.  Can  the  prototype  MCAS  be  reduced  through  either  the  elimination  of 
redundant  questions  or  combination  of  such  questions? 

4.  Are  there  discemable  differences  between  squadrons  surveyed,  and  does  there 
appear  to  be  any  consensus? 

5.  Does  the  maintenance  data  support  the  components  /  structure  of  the 
maintenance  adapted  MOSE? 

D.  SCOPE  AND  LIMITATIONS 

The  intent  of  this  study  is  to  gain  a  better  understanding  of  the  Naval  Aviation 
maintenance  safety  climate,  understanding  that  efforts  are  underway  to  look  into  the 
supervisory  and  aircrew  factors.  Three  California  based  Reserve  Squadrons  are  used  as 
subjects  in  the  survey  due  to  their  availability,  accessibility,  close  proximity;  this  effort  is 
a  preliminary  study.  Neither  Marine  Corps  units  nor  active  duty  personnel  participated  in 
this  study,  nor  did  any  Tactical  Air  Squadrons.  Only  the  Naval  Aviation  maintenance 
personnel  responses  are  used  in  this  analysis.  Incomplete  surveys  are  omitted. 
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II.  LITERATURE  REVIEW 


A.  BACKGROUND 

It  has  been  evident  for  most  of  recorded  history  that  people  are  prone  to  commit 
errors-  “To  err  is  human.”  As  our  technology  expands,  as  our  wars  multiply,  and  as  we 
invade  more  and  more  of  nature,  we  create  systems  -  organizations,  and  the  organization 
of  organizations  -  that  increase  the  risk  for  the  operators,  passengers,  innocent  bystanders, 
and  for  future  generations.  Most  of  these  risky  enterprises  have  catastrophic  potential,  the 
ability  to  take  the  lives  of  hundreds  of  people  in  one  blow,  or  to  shorten  or  cripple  the 
lives  of  thousands  or  millions  more  (Perrow,  1984).  Catastrophic  errors  in  a  system  are 
often  the  result  of  the  gradual  aggregation  of  small  errors  across  a  system.  Considering 
organizations  as  systems,  and  as  parts  of  systems,  is  important  in  order  to  understand  how 
organizations  work,  and  how  to  best  mitigate  risk  in  them  (Grabowski  &  Roberts,  1996). 
Advances  in  technology  have  increased  the  reliability  of  most  system  components,  but  the 
percentage  of  human  error-related  incidents  and  accidents  has  remained  fairly  constant. 

B.  STUDY  FRAMEWORK 

The  theoretical  framework  for  the  construction  of  the  maintenance  questionnaire 
survey  used  in  this  study  is  derived  from  the  work  done  by  Ciavarelli  and  Figlock  (1997) 
at  the  School  of  Aviation  Safety,  Naval  Postgraduate  School,  Monterey,  CA.  •  The 
framework  is  based  on  key  attributes  of  successful  organizational  risk  management 
processes  identified  by  Libuser  (1994)  and  Roberts  (1990),  and  from  a  safety  climate 
analysis  published  by  Zohar  (1980).  The  existing  CSA  questionnaire  was  generated  using 
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academic  resources  and  professional  questionnaires,  which  provided  guidance  regarding 
design  and  construction  of  questionnaires  (Ciavarelli  and  Figlock,  1997). 

Five  major  areas  concerning  the  effectiveness  of  organizations  at  managing  risk 
are  identified  in  a  model  developed  by  Libuser  (1994).  These  areas  included  process 
auditing,  a  reward  system,  quality  of  operations,  risk  perception  and  command  and 
control.  Libuser’s  work  was  adapted  by  Ciavarelli  and  Figlock  (1997)  to  develop  a 
Model  of  Safety  Effectiveness  (MOSE)  that  incorporates  standard  Naval  aviation 
language  and  safety  practices.  The  five  MOSE  components  are  summarized  as  follows: 

1.  Process  Auditing  -  a  system  of  ongoing  checks  to  identify  a  hazardous 
condition  and  a  process  to  take  corrective  actions. 

2.  Reward  System  -  a  system  to  recognize  and  reward  safe  behaviors,  and  to 
discourage  unsafe  behaviors. 

3.  Quality  standards  -  Sets  control  procedures  to  monitor  quality  and  correct 
deviations  to  referenced  standards. 

4.  Risk  Management  -  Systematic  process  to  identify  and  manage  risk. 

5.  Command  and  Control  -  Established  policies  and  procedures  to  conduct  safe 
operations. 

A  complete  description  of  the  MOSE  components  is  provided  in  Appendix  A. 

Ciavarelli  and  Figlock’ s  (1997)  short  term  goal  is  to  provide  the  QMB  with 
feedback  on  issues  concerning  aviation  command  climate,  morale  of  Naval  Aviation 
personnel,  squadron  workload  and  resource  availability,  the  estimated  success  of  ongoing 
safety  interventions,  and  other  factors  related  to  safely  managing  Naval  flight  operations. 
Their  long-term  goal  is  to  develop,  validate  and  apply  methods  for  assessing  and 
mitigating  risk  in  organizations  that  must,  by  the  nature  of  their  business,  conduct 
hazardous  operations.  They  develop  and  apply  questionnaire  survey  methodology  to 
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assess  the  effectiveness  of  Naval  Aviation  units  in  the  management  of  risks  associated 
with  flight  operations.  This  study  is  the  first  of  several  planned  investigations  intended  to 
improve  the  understanding  about  possible  influences  of  organizational  factors  in  aircraft 
accidents,  specifically  understanding  what  possible  influences  a  Naval  Aviation 
command  may  have  on  the  chain  of  events  leading  to  an  aircraft  mishap  (Ciavarelli  & 
Figlock,  1997). 

C.  ORGANIZATIONAL  CULTURE  AND  CLIMATE 

Roberts  (1990)  examines  organizations  that  are  successful  in  managing  risks 
associated  with  hazardous  operations.  She  labels  this  type  of  an  organization  as  a  “high- 
reliability”  organization  (HRO),  which  operates  in  hazardous  environments  yet  produces 
a  very  low  rate  of  accidents  and  incidents.  Grabowski  and  Roberts  (1996)  contend  that 
strong  cultures  and  norms  that  reinforce  their  mission  and  goals  characterize  HROs.  A 
culture  is  defined  as  the  shared  values,  beliefs  and  assumptions  that  may  govern  decision 
making,  attitudes,  safety,  and  proper  conduct  within  an  organization.  Norms  are  rules  and 
regulations  that  guide  and  control  the  behavior  of  individuals  and  organizations.  They 
focus  attention  on  procedures,  policies,  and  reward  structures  that  are  consistent  with  the 
organization’s  mission.  Grabowski  and  Roberts  (1996)  find  that  various  cultures 
operating  in  large-scale  systems  are  the  glues  tying  the  segments  together. 

Zohar  (1980)  finds  strong  evidence  that  organizations  with  successful  safety 
programs  have  a  strong  management  commitment  to  safety.  She  determines  that  in  low 
accident  companies,  top  management  is  personally  involved  in  safety  activities  on  a 
routine  basis,  and  safety  matters  are  given  high  priority  in  company  meetings  and  in 
production  scheduling.  This  underlying  safety  climate  refers  to  the  shared  perception  of 
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an  organization’s  members  that  the  organization’s  leaders  are  genuinely  committed  to 
safety  of  operations.  She  contends  that  such  a  climate  results  in  increased  performance 
reliability  of  workers,  good  housekeeping,  and  high  design  and  maintenance  standards' for 
work  environments. 

D.  HRO:  CLIMATE  AND  CULTURE 

To  operate  a  complex  system  successfully,  the  human-machine  system  must  be 
supported  by  an  organizational  infrastructure  of  operating  concepts,  rules,  guidelines,  and 
documents.  In  high-risk  endeavors  such  as  aircraft  operations,  it  is  essential  that  such 
support  be  flawless,  as  the  price  of  deviations  can  be  high.  When  operating  rules  are  not 
adhered  to,  or  the  rules  are  inadequate  for  the  task  at  hand,  not  only  will  the  system’s 
goals  be  thwarted,  but  there  may  also  be  tragic  human  and  material  consequences  (Degani 
&  Wiener,  1994).  Both  Roberts  (1990)  and  Libuser  (1994)  believe  that  organizations 
operating  safely  and  effectively  have  certain  key  characteristics  in  common.  Leadership 
style,  sound  safety  management  policies,  procedure  standardization,  adequacy  of 
resources  and  staffing,  and  a  defined  system  for  risk  management  are  such  characteristics. 
In  large-scale  systems,  maintaining  or  developing  a  culture  of  safety  and  reliability  can  be 
important  in  mitigating  risk.  Oversight,  checks  and  balances,  and  strong  cultural  norms 
are  organizational  protections  and  buffers  that  reinforce  an  organization’s  goals;  they  can 
mitigate  risk  by  assuring  errors  are  caught,  that  appropriate  organizational  norms  are 
developed  and  reinforced,  and  that  the  system  improves  over  time  (Grabowski  &  Roberts, 
1996). 
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E.  AVIATION  MAINTENANCE  AND  RAMP  ACTIVITIES 


As  anyone  working  in  aviation  maintenance  knows,  there  have  been  a  number  of 
serious,  even  fatal,  accidents  over  the  years  that  were  caused  primarily  by  maintenance 
errors  (Reason  &  Maddox,  1998).  Boiled  down  to  its  essence,  the  task  of  a  maintenance 
mechanic  is  to  take  off  and  then  replace  some  of  the  three  to  four  million  removable  parts 
on  a  modem  aircraft  platform.  Much  goes  on  in  between,  of  course,  but  these  basic  steps 
of  disassembly  and  re-assembly  remain  constant  features  of  the  work  —  the  latter  attracts 
by  far  the  largest  number  of  errors  (Reason,  1990).  The  maintenance  area  is  an  unstable 
environment  in  a  number  of  different  ways:  from  the  point  of  view  of  human  factors  it 
does  not  compromise  a  well-designed  work  environment.  Problems  range  from  complex 
activity  and  congestion  in  a  poorly  delineated  environment  to  the  problems  of  controlling 
the  movement  of  large  objects  in  space  with  inadequate  information  and  cues  (McDonald 
&  Fuller,  1994).  It  is  clear  that  problems  with  human  factors  still  exist  in  aircraft 
maintenance  facilities  -  even  those  that  are  built  in  strict  accordance  with  the  prevailing 
architectural  standards  (Maddox,  1998).  Therefore,  within  maintenance  activities,  the 
human  operator’s  role  is  critical  to  safe  and  efficient  operations. 

The  technology  of  the  ground  handling  process  also  poses  interesting  problems  for 
safety.  Aircraft  are  designed  to  fly;  they  are  not  optimally  designed  for  ramp  operations. 
The  aircraft’s  size  and  location  in  space  relative  to  the  operator  invite  misperceptions  and 
misjudgments  of  distance  and  location;  the  fragile  skin  and  appendages  of  an  aircraft  are 
easily  damaged  and  such  damage  if  undetected,  can  have  disastrous  consequences  in 
flight.  The  cyclical  nature  of  the  ramp  operator’s  job  often  requires  immediate  and 
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demanding  increases  of  both  mental  and  physical  performance  from  a  resting  and  perhaps 
fatigued  level  (McDonald  &  Fuller,  1994). 

There  are  many  factors  within  a  maintenance  and  ramp  activity  that  can  be 
associated  with  failures  (Reason  &  Maddox,  1998).  These  factors  can  have  an  adverse 
effect  upon  the  working  practices  of  those  within  the  maintenance  and  ramp  areas  and 
hence,  human  performance.  Factors  within  the  workplace  are  known  as  local  errors,  and 
those  that  lie  within  the  organization,  organizational  factors.  The  organizational  factors 
create  the  local  errors. 

One  such  local  factor  is  the  knowledge,  skills  and  experience  of  maintenance 
personnel;  personnel  can  be  unfamiliar  with  a  defect  or  aircraft  type,  lack  specific  training 
or  skills  and  have  inappropriate  experience  to  perform  proper  maintenance.  A  morale 
factor  can  exist;  people  clash  in  personality,  get  frustrated,  or  can  be  unhappy  with  the 
work  situation.  Availability  of  tools,  equipment  and  parts  quality  may  be  another  factor. 
Other  local  factors  may  include  fatigue,  the  pressures  of  a  high  workload,  problems  with 
shift  patterns,  and  environmental  problems.  Unclear  manuals  and  procedures,  quality  of 
safety  equipment  and  training  may  also  be  present  (Maddox,  M,  1998). 

An  organizational  factor  may  be  organization  structures;  there  can  be  concerns 
about  restructuring  and  downsizing,  ill-defined  duties,  or  too  many  layers  of  management 
in  the  organization.  People  management,  the  lack  of  top  level  awareness  of  problems  that 
technicians  face,  and  ill-defined  career  paths,  may  be  an  issue.  Training,  operational 
pressures,  planning  and  scheduling  problems  and  communication  within  the  organization 
can  also  be  examples  of  organizational  factors.  These  factors  are  usually  present  within 
most  maintenance  facilities,  and  can  directly  impact  the  climate  within  which  the 
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maintenance  technician  or  ground  handler  must  cope.  The  maintenance  and  ramp  activity 
is  a  dangerous  work  environment  presenting  the  risk  of  death  or  disabling  injury  to  those 
involved.  There  is  an  ever-present  potential  for  unreported  ground  handling  damage  to 
aircraft  while  in  the  hangar  or  on  the  ramp.  Therefore,  policies  and  procedures  are 
needed  to  minimize  this  potential  (Maddox,  1998). 

F.  PHILSOPHY,  POLICY,  PROCEDURES  AND  PRACTICES 

Philosophy,  policy,  procedures  and  practices,  the  four  P’s,  are  recognized  as 
important  aspects  in  flight  deck  operations  (Degani  &  Wiener,  1994).  Philosophy  deals 
with  the  over-arching  view  of  how  the  organization  will  conduct  business,  including 
flight  operations.  An  organization’s  philosophy  is  largely  influenced  by  the 
organization’s  leaders,  and  also  by  its  culture.  Policies  are  broad  specifications  of  the 
manner  in  which  management  expects  operations  such  as  training,  flying,  and 
maintenance  to  be  performed.  An  organization’s  policy  embodies  the  philosophy. 

Procedures  should  be  designed  to  be  as  consistent  as  possible  with  the  policies, 
which  are  consistent  with  the  philosophy.  Procedures  exist  to  specify  what  the  task  is, 
when  the  task  is  conducted  and  by  whom,  how  the  task  is  done,  and  the  sequence  of 
actions  needed  to  perform  the  task.  Practices  are  an  extension  of  and  are  governed  by  the 
philosophy,  policies  and  procedures  within  an  organization.  A  practice  is  the  activity 
actually  conducted  on  the  flight  deck  and  it  is  the  ultimate  factor  that  determines  the 
quality  of  a  systems  output.  The  goal  of  management  is  to  promote  ‘good’  practices  by 
specifying  coherent  procedures  (Degani  &  Wiener,  1994).  Maintenance,  like  flight  deck 
operations,  is  an  activity  that  relies  heavily  on  the  four  P’s  described  above;  there  is  the 
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leadership  philosophy,  the  policies  within  the  maintenance  department,  the  Standard 
Operating  Procedures  (SOP),  and  the  technician’s  or  mechanics  practices. 

G.  ASSESSMENT  AND  INTERVENTION  OF  RISK 

Looking  at  all  the  components  within  the  MOSE  and  seeing  if  these  apply  to  the 
Naval  Aviation  maintenance  community  is  called  its  assessment.  Do  the  five  components 
of  process  auditing,  reward  system  and  safety  culture,  quality,  risk  management,  and 
command  and  control  relate  to  maintenance?  Assessing  maintenance  operations  shows 
that  indeed,  these  components  are  present.  There  is  a  need  for  the  auditing  of 
maintenance  logs  and  SOP.  A  strong  safety  culture  is  a  must;  everyone  must  be  safety 
conscious  and  identify  safety  problems.  Quality  control  is  the  heart  of  maintenance, 
maintenance  is  definitely  a  high-risk  environment,  and  leadership  must  be  present  to 
ensure  the  commitment  to  safety. 

Intervention  is  the  gathering  of  data  from  a  process,  the  attempts  to  fit  a  model  to 
the  data,  and  learning  more  about  the  process.  A  better  understanding  of  the 
organizational  factors  related  to  safely  managing  maintenance  operations  is  needed  and 
desired.  A  survey  of  the  maintenance  community,  based  on  the  MOSE  adapted  for 
maintenance,  is  one  method  of  intervention.  Surveys  are  most  appropriate  when 
information  should  come  directly  from  people  involved  with  the  process.  Surveys 
provide  data  describing  attitudes,  values,  habits,  and  background  characteristics  such  as 
age,  education  and  income  (Fink  &  Kosecoff,  1985).  Ciavarelli  and  Figlock’s  (1997) 
MOSE  can  be  adopted  to  incorporate  maintenance  terms  and  language,  thus  developing  a 
needed  tool  for  intervention. 
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H.  SUMMARY 


Organizational  culture  has  an  effect  on  the  entire  organization.  This  is  especially 
true  in  high-risk  organizations  where  potential  outcomes  of  a  system  failure  can  be 
catastrophic  and  costly.  Naval  Aviation  is  recognized  as  a  high-risk  organization,  which 
can  experience  devastation  if  something  goes  awry.  This  is  also  true  in  the  aviation 
maintenance  community.  Several  approaches  have  been  made  to  improve  the  overall 
safety  of  Naval  Aviation:  a  system  safety  analysis,  an  analysis  of  mishap  data  and  surveys 
to  improve  the  understanding  of  organizational  issues  pertaining  to  operators.  If  the 
concepts  of  systems  safety  and  organizational  accidents  are  to  be  advanced,  aviation 
management  at  all  levels  must  be  aware  of  them  (Reason,  1990).  There  is  a  need  to 
develop  a  survey  to  gain  a  better  understanding  of  the  organizational  influences  on 
maintenance  error.  In  looking  at  different  survey  methods,  a  self-administered  survey 
needs  to  be  developed  for  the  Naval  Aviation  maintenance  community. 
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III.  METHODOLOGY 


A.  RESEARCH  APPROACH 

The  intent  of  this  study  is  to  develop  a  Maintenance  Climate  Assessment  Survey 
(MCAS)  for  the  Naval  Aviation  Maintenance  community  that  captures  the  maintainer’s 
perception  and  understanding  of  the  risks  within  their  department  and  in  performing  their 
mission.  This  study  involves  the  analysis  of  data  from  a  prototype  survey  that  is  based  on 
an  existing  model  of  high-risk  organizations.  This  is  done  to  identify  factors  that  may 
improve  safety  within  aviation  maintenance.  The  data  analysis  entails  descriptive 
statistics,  cluster  analysis  and  factor  analysis. 

B.  DATA  COLLECTION 

1.  Subjects 

A  total  of  three  Naval  Air  Reserve  Force  squadrons,  two  located  at  NAS  North 
Island  and  one  at  Moffett  FAF,  are  surveyed.  The  first  is  a  Fixed  Wing  Fleet  Logistics 
Support  (VR)  Squadron  composed  of  active  duty  and  selected  reserve  personnel 
providing  seven-day-a-week  round  the  clock,  worldwide  logistics  to  support  the  Navy  and 
Marine  Regular  and  Reserve  forces.  The  second  consists' of  a  Fixed  Wing  Maritime 
Patrol  (VP)  Squadron  with  a  worldwide  theater  of  operations,  composed  of  Selected 
Reservists  and  active  duty  personnel  (Note:  This  squadron  is  decommissioning).  The 
third  squadron  consists  of  a  Rotary  Wing  Combat  Support  (HCS)  Squadron  with  a 
mission  of  Search  and  Rescue,  Combat  Support  via  the  launch  and  recovery  of  targets  and 
torpedoes  on  the  Southern  California  Offshore  Range  (SCORE). 
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The  survey  respondents  are  primarily  Navy  enlisted  personnel  who  work  in 
aviation  maintenance  departments.  Maintenance  personnel  from  the  VR  Squadron 
located  at  the  Naval  Air  Station  (NAS)  North  Island,  CA  were  used  to  test  the  reliability 
and  consistency  of  the  prototype  survey.  The  survey  was  then  administered  to  two  other 
groups  of  maintenance  personnel  from  the  VP  and  HCS  Squadrons. 

2.  Instrument 

The  prototype  MCAS  consists  of  15  demographic  and  67  maintenance  related 
questions.  Demographic  questions  included  inquiries  about  rank,  community,  shift 
worked,  total  years  of  service,  and  total  years  of  aviation  maintenance  experience.  They 
also  include  inquiries  about  unit  home  location,  rating,  age  and  current  maintenance 
qualifications.  The  demographic  questions  preserve  anonymity  of  particpants;  names, 
social  security  numbers,  work  section,  etc  are  not  asked  for.  The  MCAS  is  constructed 
using  an  existing  air  operations  oriented  Command  Safety  Assessment  survey  (CSAS)  as 
a  basis. 

Questions  from  the  CSAS  are  re-worded  to  relate  to  “Maintained”  with 
maintenance  terms.  A  focus  group  discussed  all  156  candidate  questions  to  reduce 
redundancy  and  ensure  proper  phrasing  (Appendix  B  lists  the  candidate  questions). 
Survey  questions  are  fitted  to  the  existing  components  as  outlined  in  the  current  MOSE 
and  new  areas  are  created  to  fit  any  remaining  questions.  The  MOSE  components  consist 
of  Process  Auditing,  Reward  Systems  and  Safety  Culture,  Quality,  Risk  Management,  and 
Command  &  Control.  Additional  questions,  proposed  by  current  maintenance  experts, 
are  also  incorporated  into  the  MCAS.  All  questions  are  partitioned  according  to  the 
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MOSE  components,  and  the  residual  questions  are  examined.  They  fit  under  a  sixth 
category  peculiar  to  maintenance  vice  air  operations:  Communication  /  Relationships. 

The  MCAS  questions  are  separated  into  the  MOSE  components  of  Process 
Auditing,  Reward  System  and  Safety  Culture,  Quality,  Risk  Management,  Command  and 
Control,  and  Communication  /  Relationships.  Eight  of  the  67  questions  cover  Process 
Auditing,  10  cover  the  Reward  System  and  Safety  Culture  component,  12  cover  Quality 
as  well  as  Risk  Management,  14  cover  Command  &  Control  and  11  cover  the 
Communication  /  Relationships  component.  Eight  of  the  67  questions  are  worded 
negatively,  where  a  positive  response  is  indicated  by  a  low  value.  The  survey  questions 
utilize  a  five  point  Likert  rating  scale  with  the  following  verbal  anchors:  Strongly 
Disagree,  Moderately  Disagree,  Neutral,  Moderately  Agree,  and  Strongly  Agree  (The 
MCAS  is  included  in  Appendix  C). 

3.  Procedure 

The  MCAS  is  distributed  in  person.  All  persons  taking  the  MCAS  are  told  what 
the  survey  is,  why  it  is  being  given  and  told  that  their  responses  are  completely 
anonymous.  The  Fleet  Support  Squadron  received  the  MCAS  during  a  safety  stand 
down.  Groups  of  10  to  15  people  were  administered  to  until  all  available  personnel  had 
taken  the  survey.  Each  person  was  given  approximately  15  to  20  minutes  or  more  time  if 
needed,  to  take  the  survey.  The  Patrol  Squadron  MCAS  was  given  one  on  one  going 
from  office  to  office  throughout  the  squadron’s  workspaces  over  a  drill  weekend.  The 
MCAS  was  handed  out  in  a  workspace  and  the  participants  were  individually  told  what 
the  survey  was,  why  it  was  needed  and  that  their  individual  responses  would  not  be 
reported.  The  workspace  MCAS  took  much  longer  to  administer  because  participants 
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had  a  job  to  do.  Some  MCAS  forms  were  completed  right  away,  some  were  collected  a 
few  hours  later.  Throughout  the  workday  and  the  workspaces,  the  MCAS  were  collected 
on  an  individual  basis.  The  Combat  Support  Squadron  personnel  took  the  survey  at 'the 
beginning  of  a  safety  brief  over  a  drill  weekend.  The  Squadron  personnel  were  split  into 
two  groups  of  30  to  40  people.  Each  group  was  given  approximately  15  to  20  minutes  or 
more  time  if  needed,  to  take  the  survey.  The  MCAS  was  collected  at  the  time  it  was 
completed.  For  each  squadron,  each  survey  was  hand  numbered  for  future  order. 

C.  DATA  ANALYSIS 

1.  Data  Tabulation 

Survey  demographics  and  responses  are  hand  entered  into  an  Excel  (Microsoft, 
1996)  spreadsheet.  The  spreadsheet  consists  of  rows  of  respondents  and  columns  of 
survey  questions.  The  first  fifteen  columns  are  for  demographics,  with  another  sixty- 
seven  columns  representing  the  actual  survey  questions.  Results  are  coded  in  the 
spreadsheet  by  assigning  scores  of  1,  2,  3,  4,  or  5  corresponding  to  the  Likert  scale  of 
Strongly  Disagree,  Disagree,  Neutral,  Agree  and  Strongly  Agree,  respectively.  Questions 
8,  20,  22,  39,  40,  41,  and  54  are  negative  in  wording  and  ask  respondents  to  answer 
opposite  to  how  they  respond  to  the  other  questions.  Survey  questionnaire  items  that  had 
no  response  are  left  blank  and  were  dealt  with  by  S-PLUS  (Mathsoft,  1997)  as  the  data  is 
transformed  into  a  SPLUS  4.0  data  frame  for  complete  data  analysis. 

2.  Statistical  Analysis 

Basic  summary  statistics  are  developed.  Descriptive  analysis  is  conducted  on  the 
data  to  describe  basic  and  general  information  about  the  demographic  and  question 
results.  These  results  include  the  distribution  of  survey  participants  by  rank  and  service 
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classifications,  and  the  total  sample,  mean  and  standard  deviation  for  each  of  the  67 
survey  questions.  The  S-Plus  4.0  (Mathsoft,  1997)  program  is  used  for  multivariate 
statistical  analyses.  Clustering  methodologies  such  as  agglomerative  nesting  and  divisive 
analysis  utilizing  the  S-PLUS  functions  AGNES  and  DIANA  are  used  to  answer  the  third 
research  question  with  the  aid  of  factor  analysis.  Factor  and  principal  component 
analyses  are  performed  to  answer  the  second  and  fifth  research  questions,  utilizing  the  S- 
PLUS  functions  FACTANAL  and  PRINCOMP. 
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IV.  RESULTS 


A.  DESCRIPTIVE  STATISTICS 

Descriptive  statistics  of  survey  participants  are  presented  in  Table  1.  For  each 
participating  Squadron,  Fleet  Support,  Patrol  and  Combat  Support,  there  are  two 
columns.  The  column  titled  “All”  is  representative  of  all  survey  participants  for  the 
respective  squadron.  The  column  with  the  heading  “Maint”  pertains  to  maintenance 
respondents  only.  Table  1  lists  the  numbers  of  officers,  E1/E3,  E4/E6,  and  E7  and  above 
for  each  squadron  as  well  as  the  number  of  TAR,  SELRES  and  USN  designated 
personnel. 


Fleet  Support 
Sqaudron 

Patrol 

Sqaudron 

Combat  Support 
Sqaudron 

All 

Sqaudrons 

■ 

All 

Maint 

All 

Maint 

All 

All 

Officers 

14 

1 

13 

0 

14 

0 

41 

1 

E1/E3 

7 

7 

2 

2 

7 

7 

16 

16 

E4/E6 

77 

68 

71 

67 

42 

39 

190 

174 

E7+ 

10 

10 

7 

7 

4 

4 

21 

21 

Total 

108 

86 

93 

76 

67 

50 

268 

212 

SELRES 

65 

47 

59 

45 

34 

26 

158 

118 

TAR 

42 

38 

34 

31 

32 

23 

108 

92 

USN 

1 

1 

0 

0 

1 

1 

2 

2 

Total 

108 

86 

93 

76 

67 

50 

268 

212 

Table  1.  Distribution  of  Survey  Participants. 


From  Table  1,  the  maintenance  personnel  involved  in  taking  the  survey  represent 
79  percent  of  the  total  survey  population.  Maintenance  E4/E6  enlisted  personnel 
represented  a  total  of  82  percent  of  the  data  analyzed  in  this  thesis.  This  is  important  to 
note  because  most  of  the  maintenance  work  and  work  supervision  is  done  at  this  level. 
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B.  MAINTENANCE  RESPONSE  STATISTICS 


The  MOSE  component  results  for  maintenance  respondents  are  presented  in 
Tables  2  through  7  (The  MOSE  component  results  for  all  respondents  are  tabulated  in 
Appendix  D).  Tables  2  through  7  list  the  maintenance  results  for  all  three  Squadrons  for 
each  of  the  six  MOSE  components.  The  tables  show  total  respondents  for  each  question 
(n),  average  rating  for  each  question  on  a  “1”  to  “5”  Likert  scale  and  the  standard 


deviation  for  each  question.  It  is  interesting  to  note  that  average  results  amongst  the 
squadrons  are  relatively  similar.  Also  noted  is  that  for  the  eight  questions  of  the  MCAS 
worded  negatively,  the  respondent’s  answers  reflect  as  expected. 


The  first  MOSE  component  presented  in  Table  2  is  Process  Auditing.  There  are 
eight  questions  in  this  component.  The  only  question  that  raises  a  concern  is  number  59 
(overall  mean  =  2.73).  All  three  Squadrons  rate  this  question  below  neutral,  in  the 
disagree  direction  of  the  Likert  scale.  The  question  entails  the  use  of  safety  staff  to 
manage  personnel  at  risk.  A  better,  more  effective  use  of  safety  staff  and  a  higher 
presence  of  safety  staff  within  the  squadron  may  be  a  possible  way  to  influence  this 
perspective  in  the  positive  direction.  The  remaining  eight  questions  are  answered 


positively  (mean  range  =  3.08  to  4.83). 


Question 

Fleet  Support 
Squadron 

Patrol 

Squadron 

Combat  Support 
Squadron 

Combined 

Squadrons 

Total 

Avg 

StdDev 

Total 

Avg 

StdDev 

Total 

Avg 

StdDev 

Total 

Avg 

StdDev 

1 

86 

4.42 

0.458 

76 

4.38 

0.653 

50 

4.42 

0.758 

212 

4.41 

0.623 

2 

86 

3.79 

0.897 

76 

3.79 

0.838 

50 

3.74 

1.121 

212 

3.77 

0.952 

3 

86 

4.47 

0.699 

75 

4.19 

0.766 

49 

4.33 

0.774 

210 

4.33 

0.746 

4 

86 

4.41 

0.879 

75 

4.29 

0.785 

50 

4.16 

0.889 

211 

4.29 

0.851 

5 

85 

4.60 

0.576 

76 

4.83 

0.444 

50 

4.62 

0.667 

211 

4.68 

0.562 

46 

85 

4.25 

1.093 

75 

4.11 

0.649 

50 

4.20 

0.782 

210 

4.18 

0.841 

59 

85 

2.66 

1.418 

76 

2.55 

1.136 

50 

2.98 

1.237 

211 

2.73 

1.264 

67 

85 

3.18 

1.004 

75 

3.39 

0.999 

50 

fccKlBl 

1.175 

210 

3.21 

1.059 

Table  2.  Process  Auditing  MOSE  Component. 
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The  Reward  System  and  Safety  Culture  MOSE  component  is  presented  in  Table 


3.  Ten  questions  comprise  this  component,  with  two  that  are  negatively  worded, 
questions  41  and  54.  These  two  questions  appear  to  be  a  problem,  but  reverse  coding 


shows  that  they  are  answered  positively  (mean  range  =  3.49  to  4.02,  and  3.02  to  3.45 


respectively).  The  other  eight  questions  are  all  answered  positively  (mean  range  =  3.10  to 


4.32). 


Question 

Fleet  Support 
Squadron 

Patrol 

Squadron 

Combat  Support 
Squadron 

Combined 

Squadrons 

Total 

Avg 

StdDev 

Total 

Avg 

StdDev 

Total 

Avg 

StdDev 

Total 

Avg 

StdDev 

6 

85 

3.51 

1.158 

76 

3.68 

0.996 

50 

3.82 

1.155 

211 

3.67 

1.103 

7 

85 

3.93 

1.281 

75 

3.99 

0.814 

50 

3.84 

1.167 

210 

3.92 

1.087 

18 

86 

4.24 

76 

4.17 

0.870 

50 

3.98 

1.078 

212 

4.13 

19 

85 

3.35 

1.874 

76 

4.01 

0.887 

50 

4.08 

0.922 

211 

3.82 

1.228 

36 

84 

3.37 

1.320 

74 

3.20 

1.007 

49 

3.10 

1.141 

207 

3.22 

1.156 

41 

86 

2.51 

1.406 

76 

2.24 

1.118 

50 

1.98 

1.040 

212 

2.24 

1.188 

53 

86 

3.64 

1.198 

76 

3.47 

0.945 

50 

EE[*3 

1.091 

212 

1.078 

54 

85 

2.66 

1.418 

76 

2.55 

1.136 

50 

2.98 

1.237 

211 

2.73 

1.264 

58 

85 

4.14 

1.075 

76 

4.26 

0.719 

49 

4.04 

1.172 

210 

4.15 

0.989 

60 

85 

4.31 

0.715 

76 

4.32 

0.697 

50 

4.32 

■rena 

211 

4.31 

0.775 

Table  3.  Reward  System  and  Safety  Culture  MOSE  Component. 

The  third  MOSE  component  is  Quality.  This  component  has  twelve  questions. 


with  one  question,  number  twenty  (mean  range  =  4.00  to  4.05),  requiring  reverse  coding. 


All  questions  in  this  component,  except  one,  are  answered  positively  (mean  range  =  3.45 


to  4.37).  Question  37  (overall  mean  =  2.82)  is  answered  in  a  negative  manner.  This 


question  relates  to  staffing;  Is  it  sufficient  from  shift  to  shift.  The  Quality  component 


results  are  presented  in  Table  4. 
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Table  4.  Quality  MOSE  Component. 

Risk  Management  is  the  fourth  MOSE  component  and  it  has  twleve  questions, 
with  three  questions,  numbers  12  (mean  range  =  2.84  to  3.03),  22  (mean  range  =  3.55  to 
3.88),  and  40  (mean  range  =  2.93  to  3.12),  requiring  reverse  coding.  All  but  one  of  the 
questions  were  answered  positively  (mean  range  =  2.98  to  4.33).  Question  51  (overall 
mean  =  2.78)  has  negative  responses.  This  question  concerns  equal  workloads  and  equal 
stress  /  fatigue  between  shifts.  This  negative  perception  may  be  of  some  concern.  The 
Risk  Management  component  results  are  presented  in  Table  5. 


Table  5.  Risk  Management  MOSE  Component. 

Command  and  Control  is  the  fifth  component  of  the  MOSE  model.  It  comprises 

the  largest  number  (14)  of  questions  in  the  survey,  with  question  39  worded  negatively. 
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Reverse  coding  question  39  (mean  range  =2.54  to  2.92)  shows  that  it  is  responded  to 
negatively  in  the  survey.  Question  39  deals  with  passdown  between  shifts  within  the 
command.  All  other  questions  in  this  component  were  answered  positively  (mean  range 
=  2.96  to  4.68).  Table  6  tabulates  the  results  for  the  maintenance  survey  for  the 


Command  and  Control  MOSE  component. 


Question 

Fleet  Support 
Squadron 

Patrol 

Squadron 

Combat  Support 
Squadron 

Combined 

Squadrons 

Total 

Avg 

StdDev 

Total 

Avg 

StdDev 

Total 

Avg 

StdDev 

Total 

Avg 

StdDev 

13 

86 

4.41 

0.621 

76 

4.28 

0.759 

50 

4.32 

0.713 

212 

4.33 

0.698 

14 

85 

3.92 

0.910 

75 

3.93 

0.875 

50 

4.16 

0.817 

210 

4.00 

0,867 

25 

86 

4.00 

0.918 

76 

4.05 

0.798 

50 

4.08 

0.900 

212 

4.04 

0.872 

26 

86 

3.90 

1.036 

76 

4.13 

0.660 

50 

4.04 

0.968 

212 

4.02 

0.888 

27 

86 

3.91 

0.932 

75 

4.08 

0.749 

50 

3.92 

0.966 

211 

3.97 

0.882 

28 

86 

3.94 

0.950 

76 

3.95 

0.798 

50 

3.90 

0.886 

212 

0.878 

29 

86 

3.71 

1.173 

75 

3.27 

1.212 

50 

3.76 

1.153 

211 

3.58 

1.179 

30 

86 

3.81 

1.259 

76 

4.08 

0.707 

50 

4.06 

0.998 

212 

3.98 

0.988 

38 

85 

3.82 

1.242 

76 

3.72 

1.015 

50 

3.74 

1.026 

211 

3.76 

1.094 

39 

85 

3.35 

1.541 

75 

3.08 

0.941 

50 

Kxia 

1.129 

210 

3.30 

1.204 

56 

85 

4.45 

0.703 

76 

4.47 

0.621 

50 

4.68 

0.551 

211 

4.53 

0.625 

64 

85 

4.01 

0.964 

76 

4.00 

0.766 

50 

4.16 

0.842 

211 

4.06 

0.857 

65 

84 

3.81 

1.264 

76 

3.96 

0.807 

50 

4.14 

0.904 

210 

3.97 

0.992 

66 

83 

2.96 

UiTil 

75 

3.07 

0.920 

50 

3.14 

0.990 

208 

3.06 

1.039 

Table  6.  Command  and  Control  MOSE  Component. 

The  final  component  of  the  MOSE  is  Communication  /  Relationships.  This 

component  has  eleven  questions,  with  question  eight  (mean  range  =  3.23  to  3.70)  worded 


negatively.  Questions  34  (overall  mean  =2.99)  and  43  (overall  mean  =  2.64)  show  some 


concern,  resulting  in  response  values  less  than  three.  The  Communication  /  Relationship 


component  is  tabled  in  Table  7. 
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Question 

Fleet  Support 
Squadron 

Patrol 

Squadron 

Combat  Support 
Squadron 

Combined 

Squadrons 

Total 

Avg 

StdDev 

Total 

Avg 

StdDev 

Total 

Avg 

StdDev 

Total 

Avg 

StdDev 

8 

84 

2.77 

1.575 

75 

2.39 

1.102 

50 

2.30 

1.165 

209 

2.49 

1.281 

15 

86 

3.63 

1.531 

76 

3.33 

1.038 

50 

3.28 

1.161 

212 

3.41 

1.243 

16 

85 

3.86 

0.956 

74 

3.53 

0.954 

50 

3.68 

0.794 

209 

3.69 

0.901 

31 

86 

3.73 

1.257 

76 

3.40 

1.034 

50 

3.52 

1.216 

212 

3.55 

1.169 

32 

85 

4.01 

1.083 

76 

4.34 

0.684 

50 

3.88 

1.154 

211 

4.08 

0.974 

33 

86 

3.92 

75 

3.96 

0.892 

50 

3.94 

0.998 

211 

3.94 

0.953 

!  34 

86 

2.90 

0.918 

73 

3.16 

1.028 

50 

2.90 

1.055 

209 

2.99 

1.000 

j  35 

85 

3.65 

1.612 

75 

3.43 

1.029 

50 

3.32 

1.115 

210 

3.46 

1.252 

43 

84 

2.80 

1.440 

75 

2.48 

1.178 

50 

2.64 

1.174 

209 

2.64 

1.264 

52 

86 

3.79 

1.203 

75 

3.99 

0.762 

50 

3.92 

0.900 

211 

3.90 

0.955 

57 

85 

3.94 

1.151 

76 

4.03 

0.730 

50 

4.10 

1.035 

211 

4.02 

0.972 

Table  7.  Communication  /  Relationship  MOSE  Component. 


Each  question  within  the  MOSE  is  answered  positively  for  all  participating 
squadrons.  The  average  for  all  questions  in  each  MOSE  component  was  computed  and  is 
tabulated  in  Table  8. 


MOSE  Component 

Patrol 

Squadron 

Combat  Support 

Squadron 

Combined 

Squadrons 

Process  Auditing 

3.97 

3.94 

3.94 

3.95 

Reward  System 

3.73 

3.83 

3.78 

3.78 

Quality 

3.88 

4.01 

4.05 

3.98 

Risk  Management 

3.52 

3.67 

3.63 

3.61 

Command  &  Control 

3.81 

3.85 

3.90 

3.85 

Comm/Functional  Relationships 

3.59 

3.57 

3.41 

3.56 

Table  8.  MOSE  Component  Summary  Results. 


C.  ANALYSIS  PROBLEM 

The  dataset  used  in  this  portion  of  the  analysis  uses  only  the  Maintenance  surveys. 
Of  the  total  268  surveys  collected,  212  were  from  Maintained.  These  212  responses  are 
used  to  form  a  data  matrix,  X,  to  be  studied  and  analyzed  using  the  S-Plus  statistical 
package.  If  X  is  n  by  k,  then  n  =  212,  the  total  number  of  survey  respondents  and  k  =  67, 
the  number  of  survey  questions  presented. 

The  problem  is  to  reduce  the  existing  k  MCAS  items  to  a  more  reasonable  number 
for  a  full-scale  survey.  An  acceptable  number  of  questions  for  a  survey  is  in  the  range  of 


25  to  40  questions  (Fink,  1985).  Clustering  methods  such  as  agglomerative  nesting  and 
divisive  analysis  are  sensible  tools  for  such  problems. 

These  methods  belong  to  the  family  of  hierarchical  clustering  methods  that 
produce  a  tree-like  hierarchy.  This  structure  is  most  conveniently  visualized  through  a 
dendrogram  such  as  the  one  depicted  in  Figure  6.  The  vertical  scale  is  a  distance  measure 
that  marks  the  levels  at  which  two  clusters  are  joined  to  make  one.  The  horizontal  is  a 
nominal  scale  on  which  the  item  designators  are  permuted  so  that  the  tree  arcs  do  not 
cross  one  another.  From  the  bottom  up  we  can  view  how  the  k  items  are  joined  to  form 
successive  clusters.  The  use  of  such  information  can  help  provide  a  dispassionate  guide 
for  grouping  the  67  items  into  a  more  reasonably  sized  set. 

D.  AGGLOMERATIVE  NESTING  (AGNES) 

Two  S-PLUS  algorithms  are  used,  AGNES  and  DIANA.  Agglomerative  nesting 
(AGNES)  is  discussed  first  as  it  is  the  easier  to  describe.  It  is  a  “bottom  up”  approach  so, 
initially,  there  are  (k=67)  clusters;  each  single  item  is  viewed  as  a  cluster.  First,  one 
selects  a  distance  measure.  Suppose  euclidian  distance  is  chosen  and  is  initially 
computed  as  the  distance  between  each  pair  of  questions.  One  finds  the  smallest  value  of 
this  set  and  forms  a  cluster  of  the  two  items  associated  with  it.  Now,  there  are  k-1 
clusters  instead  of  k,  and  the  distance  of  all  items  from  the  new  cluster  of  two  must  be 
computed  and  used  to  replace  the  2  *  (k-2)  distances  of  the  two  selected  items  from  all 
others  that  have  been  supplanted.  They  are  needed  no  more  but  there  is  a  choice  in 
defining  the  new  distances.  It  was  chosen  to  average  the  distances  between  all  pairs  of 
items,  one  from  each  cluster,  to  obtain  a  distance  figure  separating  two  clusters.  This 
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done,  again  the  smallest  distance  is  chosen  for  forming  the  next  new  cluster,  etc.  The 
process  continues  until  there  is  but  one  cluster  consisting  of  all  k  items.  The  user  can 
select  the  clustering  solution  that  suits  him/her.  Scanning  the  vertical  scale  is  often  useful 
for  this  purpose. 

The  agglomerative  nesting  function  called  “AGNES”  in  S-PLUS  package  is  used 
as  a  first  method  to  cluster  the  k  questions  or  variables  of  the  Maintenance  safety  survey 
into  fewer  questions.  The  S-PLUS  function  “AGNES”  is  called  using  the  command 
agnes(daisy  ( t  (X)  )  ,  diss=T)  .  The  S-PLUS  function  “daisy”  is  used  to  calculate 
the  dissimilarity  matrix  for  X.  The  “diss=T”  statement  means  that  a  dissimilarity  matrix 
is  to  be  used  in  the  computations  to  serve  as  the  distance  function.  This  choice 
corrresponds  to  the  one  described. 

Results  from  AGNES  are  then  plotted  using  the  pltree.agnes  S-PLUS  function. 
The  pltree.agnes  function  produces  a  clustering  tree  of  agglomerative  hierarchical 
clustering  when  it  is  passed  an  AGNES  object.  It  creates  a  plot  of  a  clustering  tree.  The 
leaves  of  the  tree  are  the  original  objects  or  in  this  case,  the  k  questions.  Two  branches  of 
the  tree  come  together  at  the  distance  between  the  two  clusters  being  merged.  Figure  6 
shows  the  clustering  tree  resulting  from  the  pltree.agnes  command. 
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Figure  6.  AGNES  Clustering  Tree. 

The  AGNES  function  returns  an  object  containing  the  following  items:  order, 
height,  agglomerative  coefficient,  and  a  matrix  called  merge.  Order  is  a  vector  giving  a 
permutation  of  the  original  objects  to  allow  for  plotting,  in  the  sense  that  the  branches  of 
a  clustering  tree  will  not  cross.  Height  is  a  vector  with  the  distances  between  merging 
clusters  at  the  successive  stages.  The  agglomerative  coefficient  (ac)  measures  the  amount 
of  clustering  structure  found  in  the  dataset.  More  explicitly  to  each  item  i,  denoted  by 
m(i),  the  dissimilarity  to  the  first  cluster  it  is  merged  with,  is  divided  by  the  dissimilarity 
of  the  merger  in  the  final  step  of  the  algorithm.  The  ac  value  is  the  average  of  all  the  { 1- 
m(i)}.  The  resulting  ac  value  produced  by  AGNES  is  0.5445873.  The  AGNES  object 
has  a  resulting  merge  matrix,  describing  the  merging  of  clusters  at  steps  within  the 
clustering  process.  It  provides  the  numerical  support  for  the  dendrogram. 
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Merge  is  a  (k-1)  by  2  matrix,  where  each  row  i  of  merge  describes  the  split  at  step 
k-i  of  the  clustering.  The  merge  item  returned  using  AGNES  is  important.  This  matrix  is 
a  road  map  for  reading  the  clustering  tree  plot  resulting  from  the  pltree.agnes  function. 
This  matrix  and  the  tree  plot  are  used  as  an  aid  in  clustering  the  questionnaire  items. 


First,  pairs  are  clustered  utilizing  the  interpretations  of  the  merge  matrix.  Next,  triple  and 
quad  clusters  are  identified.  Easily  identified  pairings,  triples  and  quads  are  shown  in  the 


Table  9.  Also  shown  in  Table  9  are  the  singles,  eight  questions  that  do  not  merge  with 
previously  constructed  clusters  until  high  in  the  tree. 


Sin< 

ales 

Pairings 

Triples 

Quads  | 

7 

43 

2,14 

26,27 

44,45 

10,16 

37,51 

1,9,13 

11,66,67 

44,45,46,48 

19 

52 

ESI 

29,31 

46,48 

18,24 

39,40 

6,29,31 

12,39,40 

8,20,22,41 

32 

53 

5,47 

E&jsici 

BM.ia 

10,16,38 

56,60,62 

19,35,42,55 

35 

63 

9,13 

34,36 

56,62 

42,55 

61,65 

61 ,64,65 

25,26,27,28 

66,67 

Table  9.  AGNES  Clustering. 

From  these  identified  pairings,  additional  items  were  then  added  by  using  the 
corresponding  merge  information  until  all  questions  were  covered.  Table  10  shows  the 
remaining  questions  and  how  they  were  added  to  the  initial  clusters.  It  shows  the 
questions,  identified  by  the  AGNES  algorithm,  that  are  near  in  distance. 


(1,5,9,13,17,47,56,60,62) 

(11,34,36,66,67) 

(46,48) 

(37,51) 

PA) 

(6,10,15,16,23,29,31,38) 

(8,20,22,41,54) 

(49,50) 

(39,40) 

(30,33) 

(44,45,57,58,59,61,64,65) 

(18,21,24,25,26,27,28) 

(42,55) 

Table  10.  Additional  AGNES  Clustering. 


E.  DIVISIVE  ANALYSIS  (DIANA) 

The  AGNES  algorithm  is  a  bottom  up  approach  to  hierarchical  clustering.  The 
DIANA  algorithm  reverses  this  process.  It  is  top  down  and  the  resulting  tree  need  not  be 
the  same.  The  distance  between  clusters  is  used  for  explanatory  purposes. 
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The  process  begins  with  a  single  cluster  of  all  items  and  then  considers  all 
possible  partitions  of  the  items  into  two  clusters.  (It  is  convenient  to  think  of  it  in  this 
way,  although  efficient  algorithms  exist  which  do  not  actually  do  this.)  The  pair  of 
clusters  selected  is  the  one  with  the  greatest  separation.  Then  the  process  is  repeated  using 
each  of  the  clusters  so  formed,  etc.  Ultimately,  the  cluster  will  consist  entirely  of  single 
items.  The  dendrogram  for  this  method  appears  in  Figure  7.  Comparing  Figure  6  and 
Figure  7  it  is  seen  that  the  trees  are  quite  similar.  This  is  a  comforting  check. 

The  divisive  analysis  function  called  “DIANA”  in  the  SPLUS  package  is  used  as 
an  approach  to  “cluster”  the  k  questions  into  a  tree.  The  DIANA  function  in  S-PLUS  is 
called  using  diana  (daisy  (t  (X)  )  ,  diss=T)  ,  where  X  represents  the  data  matrix 
used  in  AGNES.  Again,  the  S-PLUS  function  “daisy”  is  used  to  calculate  the 
dissimilarity  matrix  for  X.  The  “diss=T”  statement  means  that  the  same  dissimilarity 
matrix  is  to  be  used  in  the  distance  computations. 

Results  from  DIANA  are  then  plotted  using  the  pltree.diana  function  to  look  at  the 
clustering  results.  The  S-PLUS  function  pltree.diana  creates  a  plot  of  a  clustering  tree  or 
dendrogram  given  a  DIANA  object.  The  leaves  of  the  tree  lead  to  the  k  original  objects  or 
questions.  A  branch  splits  up  at  the  diameter  of  the  cluster  being  split.  Figure  7  shows  the 
resulting  tree  of  “clusters”  from  the  DIANA  analysis. 
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Figure  7.  DIANA  Clustering  Tree. 

The  DIANA  function  returns  the  following  items:  “order”,  “height”,  the  “divisive 
coefficient”,  and  “merge”.  Order  is  a  vector  giving  a  permutation  of  the  original  objects  to 
allow  for  plotting,  in  the  sense  that  the  branches  of  a  clustering  tree  will  not  cross.  Height 
is  a  vector  with  the  diameters  of  the  clusters  prior  to  splitting.  The  divisive  coefficient 
measures  the  amount  of  clustering  found.  More  explicitly,  for  each  item  i,  denoted  by 
d(i),  the  diameter  of  the  last  cluster  to  which  it  belonged  (  prior  to  being  split  as  a  single 
object ),  divided  by  the  diameter  of  the  whole  set.  The  dc  measure  is  the  average  of  all 
{ l-d(i)}.  The  divisive  coefficient  resulting  from  DIANA  is  found  to  be  0.6559955.  Like 
the  AGNES  object,  the  DIANA  object  has  a  resulting  merge  matrix,  describing  the 
merging  of  clusters  at  steps  within  the  clustering  process. 

Again,  the  merge  matrix  and  the  tree  plot  are  used  as  an  aid  in  clustering  the 
questionnaire  items.  First,  pairs  clustered  are  identified  and  analyzed  to  see  if  they  make 
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sense.  Next,  triple  and  quad  clusters  are  identified.  Easily  identified  pairing  are  shown  in 
the  Table  11. 


Pairings 

Triples 

Quads 

19 

2,14 

WE&EM 

30,33 

56,60 

12,40,43 

11,34,37,51 

23 

3,4 

WBsBSM 

37,51 

57,58 

10,16,38 

20,22,41 1 

19,42,55,53 

35 

5,47 

42,55 

15,29,31 

30,32,33 

25,26,27,28 

39 

9,13 

WBSM 

17,18,24 

44,45,46,48 

43 

10,16 

W5EM 

46,48 

61 ,62,64,65 

53 

11,34 

29,31 

49,50 

Table  11  DIANA  Clustering. 


From  these  identified  pairings,  additional  items  are  then  added  using  the 


corresponding  merge  information  until  all  questions  are  covered.  Table  12  shows  the 
remaining  questions  and  where  they  are  added  to  the  initial  pairings.  It  shows  the 


questions,  identified  by  the  DIANA  algorithm,  that  are  near  in  distance. 


(57,58,59,61,62,63,64,65) 

(8,20,22,41,54) 

(6,10,16,23,38) 

(3,4) 

1(11,12,34,37,40,43,57) 

(15,29,31,35,36,39,66,67) 

(30,32,33,52) 

(2,7,14) 

(49,50) 

(21 ,25,26,27,28) 

Table  12.  Additional  Clustering  with  DIANA. 


Comparing  the  two  S-PLUS  methods  of  AGNES  and  DIANA  shows  some 
interesting  aspects.  First,  by  comparing  the  two  clustering  trees,  the  shape  of  each  tree  is 
very  similar.  The  initial  pairings  found  in  the  two  methods  are  alike  also.  Of  the  23 
initial  pairings  from  DIANA  and  the  21  pairings  from  AGNES,  17  are  common  to  both. 
Moving  up  the  clustering  tree  one  level,  more  similarities  are  seen.  Furthermore,  by 
looking  at  higher  levels  of  the  two  clustering  trees,  clusters  of  three  or  four  questions 
form.  Table  13  shows  the  common  pairings,  triples  and  quads  found  between  the  two 
methods. 
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r  Pairings 

mmm 

Quads 

2,14 

9,13 

46,48 

1,9,13 

25,26,27,28 

3,4 

10,16 

29,31 

42,55 

49,50 

10,16,38 

8,20,22,41 

5,47 

18,24 

■KMklcM 

44,45 

EIESX3D 

44,45,46,48 

61,65 

66,67 

Table  13.  Common  Clusters  Between  DIANA  and  AGNES. 


F.  PRINCIPAL  COMPONENTS  /  FACTOR  ANALYSIS 

The  Principal  Component  /  Factor  Analysis  material  will  help  complete  the 
questionnaire  size  reduction  process.  Principal  Component  Analysis  was  conducted  on 
the  dataset  using  the  S-PLUS  software  package.  The  S-PLUS  command  of 
princomp(X,scores=T,cor=T)  was  used  in  the  analysis.  X  is  the  212  by  67  matrix  of 
survey  responses  and  “cor=T”  means  that  the  principle  component  analysis  is  to  be  based 
on  the  correlation  matrix.  The  resulting  principle  component  output  was  plotted  using  a 
screeplot,  a  special  S-PLUS  barplot  function  for  the  class  princomp.  The  S-PLUS 
function  screeplot(prin2.out)  was  used.  It  shows  the  contribution  of  each  factor  to  the 
total  variance.  Figure  8  shows  the  resulting  screeplot. 
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Figure  8.  Principal  Component  Screeplot. 

Above  each  bar  in  the  plot  are  the  cumulative  fractions  of  the  variance.  The 

variance  is  sharply  the  highest  on  the  first  component  and  spread  rather  evenly  over  the 
remaining  components.  This  shows  evidence  that  the  survey  data  is  dominated  by  one 
dimension.  Much  of  the  variability  is  captured  in  the  first  component.  This  and  Factor 
Analysis  is  used  to  assist  in  answering  research  questions  three  and  five. 

Factor  analysis  on  the  data  is  performed  using  S-PLUS.  The  S-PLUS  command 
of  factanal(X,factors=6,method=”principal”,  rotation=”varimax”)  was  used.  Again,  X  is 
the  212  by  67  matrix  of  survey  responses,  six  factors  were  fit,  using  varimax  rotation  as 
the  method  of  aligning  the  factors  in  a  six  dimensional  space.  The  “factors=6”  input 
specifies  the  number  of  factors  in  the  solution.  An  object  of  class  factanal  results. 

The  factanal  class  results  from  fitting  factor  analysis  models.  This  class  has 
certain  properties  that  are  relevant  to  analyze.  An  important  property  is  “loadings.” 
Loadings  is  an  orthogonal  matrix  that  gives  the  loadings  for  each  factor,  each  column 
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being  linear  combinations  of  the  columns  of  X  for  the  corresponding  factor.  Another 
property  of  the  factanal  class  is  uniqueness.  The  variance  of  each  item  is  decomposed 
into  the  part  that  is  common  to  the  factor  solutions  and  the  part  that  is  unique  to  that  item. 
The  loadings  and  uniquenesses  for  each  question  in  the  corresponding  components  of  the 
MOSE  are  presented  in  Tables  14  through  19. 


Question 

Factor  1 

Factor  2 

Factor  3 

Factor  4 

Factor  5 

Factor  6 

Uniqueness 

1 

1  0.462 

0.306 

0.149 

0.176 

0.6381 

2 

0.337 

0.278 

0.189 

0.219 

0.263 

0.6553 

3 

0.525 

0.281 

-0.133 

0.102 

0.268 

-0.147 

0.5263 

4 

0.452 

0.214 

0.131 

0.230 

-0.140 

0.6555 

5 

0.478 

0.107 

0.167 

0.195 

0.6862 

46 

0.612 

0.117 

0.146 

0.5897 

59  [ 

0.234 

0.596 

0.243 

0.133 

0.148 

0.4885 

67  | 

0.143 

0.339 

0.436 

0.152 

0.6506 

Table  14.  Factor  Loadings  for  Process  Auditing  Component. 


Question 

Factor  1 

Factor  2 

Factor  3 

Factor  4 

Factor  5 

Factor  6 

Uniqueness 

6 

0.273 

0.216 

0.378 

0.207 

6.338 

0.5693 

7 

0.401 

0.287 

0.180 

0.267 

0.322 

0.5456  j 

18 

0.459 

0.313 

0.297 

0.384 

0.4447 

19 

0.366 

0.128 

0.139 

0.8288 

36 

0.314 

0.117 

0.329 

0.441 

-0.160 

0.5576  | 

41 

-0.402 

-6.213 

0.116 

-0.253 

-0.455 

0.5007 

53 

0.239 

0.131 

0.202 

0.277 

-0.105 

0.7925 

54 

-0.179 

-0.188 

-0.372 

-0.165 

6.7532 

58 

0.249 

0.593 

0.152 

0.310 

0.149 

0.4392 

60 

0.282 

0.625 

0.112 

0.204 

0.224 

0.133 

0.4073 

Table  15.  Factor  Loadings  for  Reward  and  Safety  Culture. 
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Question 

Factor  1 

Factor  2 

9 

0.507 

0.207 

10 

0.440 

0.141 

17 

0.617 

0.221 

20 

-0.222 

-0.327 

37 

0.118 

42 

0.276 

0.147 

44 

0.520 

0.239 

45 

0.469 

0.159 

47 

0.657 

0.188 

48 

0.567 

0.115 

49 

0.425 

0.215 

50 

0.401 

0.333 

Factor  3 


0.200 


0.373 


0.116 


0.537 


0.412 


0.124 


0.154 


0.461 


0.238 


0.410 


Factor  4 


0.133 


-0.258 


0.121 


0.189 


0.143 


0.250 


0.205 


Factor  5 


0.227 


0.284 


0.250 


0.184 


Factor  6  |  Uniqueness 


0.5898 


0.214 


-0.100 


0.101 


-0.464 


0.121 


0.245 


0.181 


0.152 


0.152 


0.5573 


0.4796 


0.5402 


0.6469 


0.6564 


0.5830 


0.6991 


0.5148 


0.4258 


0.6017 


0.4908 


Table  16.  Factor  Loadings  for  Quality. 


Question  Factor  1  Factor 2  Factor 3  Factor 4  Factors  Factor 6  Uniqueness 

11  _ 0.129  0.343 _ 0.114  -0.275  0.7637 

12  0.166  -0.365  0.8236 

21  0.572  0.158 _ 0.226  0.256 _ 0.5284 

22  -0.243  -0.146  -0.293  -0.473  0.6092 


23  0.162  |  0.252  0.149 _ 0.363  0.144  0.7275 

24  0.487  0,33 _ 0.415  0.194 _ 0.4371 

40 _ -0,128 _ -0.445  0.7717 

51 _ 0,496  0.178 _ 0.101  0.6977 

55 _ 0,19  0.33  0.414  -0.112  0.193  0.312  0.5364 

61  _ 0,239  0.638  0.321  0.157  0.176 _ 0.3709 

62  0.365  0.591  0.305  0.135  0.192  0.12  0.3550 

63  0.372  0.418  0.225  0.246  0.178  0.167  0.5159 


Table  17.  Factor  Loadings  for  Risk  Management. 


Question 

Factor  1 

Factor  2 

Factor  3 

Factor  4 

Factor  5 

Factor  6 

Uniqueness 

i  13 

0.483 

0.240 

0.158 

0.272 

0.183 

0.5747 

14 

0.415 

0.280 

0.287 

0.101 

0.256 

0.104 

0.5800 

25 

0.330 

0.301 

0.165 

0.553 

0.131 

0.109 

0.4384 

26 

0.328 

0.338 

0.258 

0.609 

0.3318 

27 

0.417 

0.155 

0.218 

0.642 

0.166 

0.3063 

28 

0.376 

0.327 

0.206 

0.563 

0.147 

0.3690 

29 

0.338 

0.198 

0.213 

0.176 

0.523 

0.4972 

30 

0.388 

0.248 

0.316 

0.312 

0.246 

0.165 

0.5029 

38 

0.378 

0.131 

0.335 

0.454 

0.131 

0.5032 

39 

0.107 

-0.108 

-0.372 

0.8253 

56 

0.314 

0.485 

0.229 

0.183 

0.108 

0.195 

0.5305 

64  i 

0.238 

0.609 

0.211 

0.231 

0.193 

0.4305 

65 

0.210 

0.587 

0.387 

0.141 

0.103 

0.4293 

66 

0.158 

0.196 

0.579 

0.122 

0.5808 

Table  18.  Factor  Loadings  for  Command  and  Control. 
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Question 

Factor  1 

Factor  2 

Factor  3 

Factor  4 

Factor  5 

Factor  6 

Uniqueness 

8 

-0.160 

-0.166 

-0.107 

-0.268 

-0.101 

-0.295 

0.7663 

15 

0.136 

0.264 

0.337 

0.159 

0.521 

0.120 

0.4876 

16 

0.449 

0.263 

0.199 

0.441 

0.4908 

31 

0.206 

0.226 

0.248 

0.279 

0.621 

0.3735 

32 

0.339 

0.142 

0.252 

0.307 

0.122 

0.6877 

33 

0.336 

0.417 

0.156 

0.372 

6.191 

0.5109 

34 

0.122 

0.232 

0.320 

0.317 

-0.145 

0.7043 

35 

0.300 

6.348 

0.7644 

!  43 

0.166 

0.9669 

52 

0.441 

0.181 

0.308 

0.208 

0.6322 

57 

0.185 

0.585 

0.122 

0.195 

0.220 

0.228 

0.4701 

Table  19.  Factor  Loadings  for  Communication  /  Relationships. 


For  each  component,  the  questions  having  the  maximum  loading  over  the  six 
factors  are  analyzed.  As  true  in  the  principle  component  analysis,  there  is  one  factor. 
Factor  one,  which  had  the  most  loadings  having  the  maximum  value.  This  analysis,  with 
results  shown  in  Table  20,  shows  the  relationships  between  the  factors  resulting  from  the 
factor  analysis  and  the  MOSE  components.  Of  the  67  questions,  25  of  them  load  the 
highest  in  factor  one.  This  provides  quantitative  information  on  these  items  that  are 
represented  in  the  single  most  important  factor  dimension. 


Factor  1 

Factor  2 

Factor  3 

Factor  4 

Factor  5 

Factor  6 

fcfcklrUiNiUM 

1,2,3,4,5,46 

59 

67 

Component  2 

7,18,19 

58,60 

6 

54 

36,53 

41 

9,10,17,44,45,47,48,49 

37,42,50 

20 

21,24 

61,62,63 

11,51,55 

23 

12,22,40 

EJlllfMlMllffl 

13,14,30 

56,64,65 

39,66 

25,26,27,28 

29,38 

|  Component  6 

16,32,52 

33,57 

15 

34 

15,16,31,34,35,43 

8 

Table  20.  Questions  and  Factors  Loadings  Broken  Out  by  MOSE  Component. 


Table  21  shows  the  distribution  of  the  questions  over  the  Factors  and  MOSE 
components  resulting  from  the  factor  analysis.  Interesting  to  note  is  the  number  of 
questions  that  belong  to  factor  1.  Twenty  five  of  the  67  questions  in  the  MCAS  fall  into 


the  first  factor.  The  remaining  factors  have  loadings  with  various  signs  and  provide 
contrast  type  information  about  the  items  that  dominate  them. 


Factor  1 

Factor  2 

Factor  3 

Factor  4 

Factor  5 

Factor  6 

6 

1 

1 

0 

0 

0 

8 

3 

2 

1 

1 

2 

1 

10 

8 

0 

3 

0 

0 

1 

12 

2 

3 

3 

0 

1 

3 

12 

3 

3 

1 

4 

2 

1 

14 

3 

2 

0 

1 

4 

1 

11 

1  Factor  Totals 

25 

11 

9 

6 

9 

7 

67 

Table  21.  Distribution  of  Questions  From  Factor  Analysis. 


Also  interesting  to  study  is  the  cumulative  distribution  function  of  the 
uniquenesses  for  each  question  resulting  from  the  factor  analysis.  This  is  important  to 
study  because  the  questions  with  low  uniqueness  in  the  large,  have  their  content 
represented  by  other  questions.  The  questions  with  high  uniqueness  have  individual 
dominating  status.  From  this,  one  may  be  able  to  identify  questions  that  can  be 
eliminated  because  they  may  be  non-contibutors.  Also,  we  can  identify  questions  of  high 
uniqueness,  which  either  should  be  retained  as  singletons,  or  identified  as  dominating  the 
group  to  which  they  belong.  Figure  9  is  the  resulting  S-PLUS  CDFDISC  function  plot  of 
the  sorted  uniquenesses  and  associated  probabilities  (Appendix  E  shows  the  S-PLUS 
CDFDISC  function  code). 
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Figure  9.  Question  Uniqueness  Empirical  CDF. 

Looking  at  the  plot  above  gives  a  good  view  of  the  uniqueness  for  the  questions. 

Note  the  flat  spot  appearing  in  Figure  9  at  the  0.4  uniqueness  level  on  the  x-axis.  The 
questions  relating  to  these  uniqueness  values  are  possible  candidates  for  elimination  since 
they  all  have  relatively  low  uniqueness.  Below  is  Table  22  showing  the  seven  questions 
whose  uniqueness  fall  near  or  below  the  0.4  value. 


Question 

Uniqueness 

26 

0.3318 

27 

0.3063 

28 

0.3690 

31 

0.3735 

60 

0.4073 

61 

0.3709 

62  | 

0.355 

Table  22.  Questions  with  Low  Uniquenesses. 

There  is  also  a  flat  area  Figure  9  at  the  0.75  uniqueness  level  on  the  axis 

identifying  questions  with  high  uniqueness  relative  to  the  entire  set.  All  uniquenesses  to 
the  right  of  0.75  relate  to  those  having  a  relatively  high  uniqueness.  There  are  ten  such 
questions.  Table  23  identifies  the  ten  questions  showing  the  uniqueness  values  for  each. 
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Question 

Uniqueness 

8 

0.7663 

11 

0.7637 

12 

0.8236 

19 

0.8288 

35 

0.7644 

39 

0.8253 

40 

0.7717 

43 

0.9669 

53 

0.7925" 

54 

0.7532 

Table  23.  Questions  with  High  Uniqueness. 

All  of  this  information  is  useful  to  aid  the  researcher  in  making  informative 

decisions  about  the  items  in  the  data.  Each  of  the  above  areas  of  analysis  are  used  in 
conjunction  with  one  another  to  obtain  the  goal  of  a  reduced  set  of  questions  to  be  used  in 
an  improved  and  modified  MCAS. 

G.  QUESTION  ANALYSIS 

The  S-Plus  functions  AGNES,  DIANA,  PRINCOMP  and  FACTANAL  assist  the 
researcher  in  making  decisions  or  objective  choices  about  the  items.  These  will  be 
viewed  for  reasonableness  based  on  subjective  knowledge.  The  stability  of  these  choices 
and  the  results  from  the  analysis  are  also  influenced  by  the  relatively  small  sample  size 
involved  in  this  study.  With  this  in  mind,  the  questions  are  studied  using  the  computer- 
aided  output  from  the  previous  sections.  The  prototype  MCAS  had  questions  that 
required  reverse  coding.  It  is  recommended  these  questions  be  rephrasedto  read  in  a 
positive  aspect.  (Appendix  H  lists  these  questions  and  the  recommended  modifications.) 

Decision  rules  need  to  be  identified  as  to  how  the  downsizing  of  the  MCAS  is 
approached.  First,  the  seven  questions  identified  with  low  uniqueness  are  removed. 
Likewise,  questions  identified,  as  having  high  uniqueness,  will  be  retained.  Secondly,  all 
pairs  that  are  the  same  in  both  AGNES  and  DIANA  are  studied  for  combination  or  to  see 
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if  the  questions  can  be  reworded  to  make  one  out  of  the  two.  This  done,  the  next  step  is 
to  take  all  triples  and  then  quadruples  identified  by  the  clustering  algorithms  and  look  for 
sensible  combinations  or  removal  of  questions.  At  the  completion  of  this,  the  remaining 
questions  that  have  not  been  grouped  or  clustered  will  be  studied  for  further  sensible 
reduction. 

Questions  having  low  uniqueness  are  eliminated.  There  are  seven  such  questions 
that  have  been  identified  in  Table  22.  The  elimination  of  these  seven  questions  results  in 
a  remaining  pool  of  60  questions.  Furthermore,  we  retain  the  ten  questions  identified  in 
Table  23  that  have  high  uniqueness.  This  leaves  50  questions  for  continued  analysis. 
Next,  the  cluster  analysis  results  from  AGNES  and  DIANA  are  used  for  combining 
questions  that  are  answered  in  similar  ways  or  show  redundancy. 

Questions  paired  up  that  are  common  to  both  the  AGNES  and  DIANA  methods 
are  analyzed  to  see  if  the  pairings  can  be  combined  easily.  Here,  questions  are  paired  up 
due  to  their  relative  closeness  resulting  from  the  computer-aided  output.  Questions  are 
studied  and  if  sensible,  can  be  joined  with  the  conjunction  “and”  or  reworded  to  capture 
the  intent  of  the  original  questions.  For  example,  the  first  pairing  from  Table  13  is  the 
pair  (9,13).  By  rewording  these  questions  slightly,  a  new  question  results  that  replaces 
the  two  questions  with  one. 

This  process  is  done  for  all  identified  clusters  in  Table  13  as  well  as  possible 
clusters  from  Tables  9  and  11.  Table  24  lists  these  groupings.  Appendix  F  gives  the 
suggested  rewording  of  these  newly  formed  questions  to  be  included  in  the  suggested 
MCAS. 
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Nineteen  questions  are  suggested  as  candidates  for  combining,  rewording  or 
grouping  that  reduce  the  MCAS  question  total  by  an  additional  24  questions.  A  sensible 
pairing  is  made  from  two  questions  having  high  uniqueness.  DIANA  identifies  pair 
(19,53)  as  a  cluster  and  AGNES  shows  agreement  in  that  the  dendrogram  has  the  pairing 
in  close  proximity  to  one  another.  This  pair  belongs  to  the  same  MOSE  component  and 
the  two  questions  have  the  same  intent,  thus  the  two  are  combined  into  one.  Question  1 1 
also  had  high  uniqueness  but  it  was  sensible  to  cluster  it  as  noted  in  Table  24. 


1,9,13 

1 1 ,36,59,67 

20,22 

49,50 

2,14 

10,66 

30,33 

52,56 

3,4 

16,38 

42,55 

5,47 

18,21,24 

37,51 

64,65  | 

6,23 

19,53 

45,46,48 

Table  24.  Suggested  Question  Groupings. 


Nine  singletons  result  in  this  process.  These  questions  neither  clustered  with 
others  nor  seemed  reasonable  to  combine  with  any  other  question  in  the  MCAS.  These 
singletons  are  presented  below  in  Table  25. 


7 

25 

41 

15 

29 

44 

17 

34 

63 

Table  25.  Original  Questions  Remaining. 

By  combing  the  results  in  the  first  step,  deleting  the  questions  with  relatively  low 
uniqueness  and  then  combining  questions  using  the  cluster  analysis  results,  the  MCAS 
reduces  to  35  questions;  seven  from  the  high  uniqueness,  19  from  the  clustering  or 
rewording,  and  an  additional  nine  as  singletons.  There  is  a  judgement  call  here,  and 
given  the  data  and  the  questions,  this  seems  to  be  as  far  as  one  can  go  without  adding 
additional  scrutiny  to  the  study.  Table  26  gives  the  suggested  mapping  of  the  67-question 
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prototype  MCAS  to  a  suggested  35-question  MCAS.  The  P-Type  column  contains  the 
ordinal  number  of  the  item  in  the  original  67-question  MCAS,  and  the  Mapped  column 
displays  the  ordinal  number  in  the  suggested  35-question  MCAS. 


E5MJ 

EEB333 

EB753 

LJu'iiU 

CS5330 

tAi'ijLl 

EBEEU 

E5S!S 

Eki*J3!5ll 

[— 

■OEM 

■EH 

■£H 

Deleted 

25 

{■EH 

i sni 

Deleted 

1  2 

1 

12 

9 

22 

15 

■9 

Deleted 

42 

26 

52 

31 

62 

Deleted 

3 

2 

13 

6 

23 

16 

Kl 

19 

■pg 

27 

53 

14 

63 

34 

4 

2 

14 

1 

24 

13 

34 

20 

28 

54 

32 

64 

35 

5 

3 

15 

10 

25 

17 

35 

21 

45 

29 

55 

26 

65 

35 

6 

16 

16 

11 

26 

Deleted 

36 

8 

46 

29 

56 

31 

66 

7 

7 

4 

17 

12 

27 

Deleted 

37 

22 

47 

3 

57 

33 

67 

8 

8 

5 

18 

13 

28 

Deleted 

38 

11 

48 

29 

58 

33 

9 

6 

19 

14 

29 

18 

39 

23 

49 

30 

59 

8 

10 

7 

20 

15 

30 

19 

40 

24  . 

50 

30 

60 

Deleted 

Table  26.  Prototype  MCAS  Questions  Mapped  to  Suggested  MCAS  Questions. 


50 


V.  CONCLUSION 


A.  FINDINGS 

The  existing  MOSE  can  be  used  to  model  or  capture  the  maintained  s  perspective, 
with  modifications  presented  in  this  thesis.  The  close-ended  questions,  after  tailoring 
them  towards  the  maintained  conformed  to  the  existing  MOSE  components  quite  readily. 
However,  an  additional  component.  Communication  /  Relationships,  was  added  to  make 
the  MOSE  maintenance  specific.  Results  of  the  prototype  MCAS  were  positive  in  nature 
for  all  six  of  the  adapted  MOSE  components  except  for  a  few  questions  that  raised 
concern  and  these  are  listed  in  Table  27.  These  questions  may  possibly  identify  areas  that 


need  strengthening  within  the  squadrons. 


Question 

Fleet  Support 
Squadron 

Patrol 

Squadron 

Combat  Support 
Squadron 

Combined 

Squadrons 

59 

2.66 

2.55 

2.98 

2.73 

37 

2.63 

3.07 

2.76 

2.82 

51 

2.54 

3.18 

2.64 

2.78 

39 

2.65  . 

2.92 

2.54 

2.7 

34 

2.9 

3.16 

2.9 

2.99 

43 

2.8 

2.48 

2.64 

2.64 

Table  27.  Questions  That  Raise  Concern. 

There  are  no  discemable  differences  identified  between  the  squadrons  that  were 


surveyed.  A  consenus  between  all  participating  squadrons  was  that  of  favorable  results 

for  all  MOSE  components.  Agglomerative  nesting  and  Divisive  Analysis  together  with 

Prinicpal  Components  and  Factor  Analysis  were  tools  used  to  identify  clusters  of 

questions  that  were  similar.  Although  the  Factor  Analysis  shows  that  most  of  the 

questions  load  on  factor  one,  there  is  no  evidence  against  the  structure  of  the  maintenance 

adapted  MOSE.  Redundant  questions  were  eliminated  and  questions  with  similar  intent 

were  combined.  Thus,  the  prototype  MCAS  was  reduced  from  67  to  35  questions, 
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shortening  the  survey  to  a  more  practical  length,  requiring  less  time  to  complete.  The 
finalized  MCAS  is  listed  in  Appendix  I. 

B.  OTHER  AREAS  OF  CONCERN 

There  are  numerous  future  studies  that  can  further  this  thesis.  One  such  task 
would  be  to  alter  the  questionnaire  in  order  to  use  it  with  the  Marine  Corps,  using 
Marine  specific  terms.  Also,  this  study  could  be  further  investigated  by  surveying 
active  duty  Naval  Aviation  Squadrons  to  make  comparisons  of  the  results  with  the 
Reserve  Squadron  results  presented  in  this  study.  The  Marine  Squadron  results  could  be 
compared  to  Navy  Squadrons  to  see  if  there  is  a  statistical  difference  between  the  two. 
Another  task  would  be  to  administer  the  survey  to  Regular  active  duty  Squadrons  and  do 
a  comparison  between  Reserve  and  active  duty  Squadrons.  Also,  other  analysis  methods 
could  be  used  to  validate  this  study  and  its  results. 

C.  SUMMARY 

The  Class  A  flight  mishap  rate  has  leveled  off  over  the  past  decade.  Human 
Factors  studies  are  increasing  as  they  tyy  to  identify  additional  areas  where  intervention 
could  lead  to  further  reductions  of  the  Class  A  flight  mishap  rate.  This  thesis  is  part  of  a 
Human  Factors  related  study  specifically  aimed  at  Maintenance  operations  within  Naval 
Aviation.  The  resulting  survey  is  to  be  used  as  a  tool  for  commanders  to  assess  the 
effectiveness  of  Naval  Aviation  maintenance  operations  in  the  management  of  risk. 
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APPENDIX  A.  EXISTING  MOSE  COMPONENTS 


COMPONENT  1:  PROCESS  AUDITING 

1 .  Conducts  periodic  safety  reviews  and  inspections. 

2.  Conducts  timely  reviews  and  updates  of  safety  practices  and  operating  standards. 

3.  Uses  a  systematic  process  to  set  training  goals  and  to  review  flight  qualifications. 

4.  Uses  Human  Factors  Councils  and  Boards  to  identify  and  screen  high-risk  aviators. 

5.  Makes  effective  use  of  Flight  Surgeon  for  assessing  significant  stress  reactions,  and 
possible  unsafe  attitudes  of  individuals  as  well  as  routine  medical  screening. 

COMPONENT  2:  REWARD  SYSTEM 

1.  Recognizes  safety  achievement  through  social  praise,  and  formal  awards  and 
incentives. 

2.  Safe  behavior  is  reinforced  as  a  cultural  norm  (by  command  emphasis  and  peer 
pressure). 

3.  Takes  timely  action  (s)  to  appropriately  discipline  unsafe  behavior  and  attitudes. 

4.  Encourages  everyone  to  be  safety  conscious  and  to  identify  safety  problems. 

5.  Encourages  everyone  to  report  safety  discrepancies  without  fear  of  negative 
repercussions. 

COMPONENT  3:  QUALITY 

1 .  Sets  high  quality  standards  as  an  organizational  goal. 

2.  Publishes  quality  standards  and  quality  control  procedures. 

3.  Monitors  quality  and  corrects  deviations  from  established  quality  standards. 

4.  Gains  reputation  for  high-quality  performance  in  comparison  to  reference  standards  or 
comparable  organizations. 

COMPONENT  4:  RISK  MANAGEMENT 

1 .  Has  accurate  perception  of  actual  operational  mission  risks. 

2.  Uses  a  systematic  method  for  managing  risk. 

3.  Risks  of  mission  operations  are  acknowledged  and  minimized. 

4.  Safety  risk  decisions  are  made  at  the  proper  level  (by  most  qualified  people). 

5.  Resources  (time,  budget,  staffing,  and  equipment)  are  adequate  for  performance  of 
mission  and  organizational  workload. 

6.  Does  not  compromise  safety  to  get  the  work  done  or  to  accomplish  the  mission. 
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COMPONENT  5:  COMMAND  AND  CONTROL 


1 .  Supervisors  openly  demonstrate  a  commitment  to  safety. 

2.  Supervisors  clearly  communicate  safety  goals  and  policies. 

3.  Supervisors  are  directly  involved  in  safety  management. 

4.  Safety  is  considered  as  an  integral  part  of  mission  accomplishment. 

5.  The  command  establishes  and  uniformly  enforces  operating  standards. 

6.  There  is  frequent  interaction  between  command  supervisors  and  staff. 

7.  There  is  good  communication  flow  up  and  down  the  command  chain. 

8.  Safety  training  is  emphasized  throughout  the  command. 

9.  The  status  and  respect  for  the  safety  officer’s  position  are  high. 

10.  Personnel  turnover  is  not  high  enough  to  affect  a  command's  ability  to  maintain 
"corporate"  knowledge  and  the  core  cultural  values  of  the  organization. 

1 1 .  Supervisors  provide  adequate  guidance  and  counseling  on  safety  matters. 

12.  Formal  rules  and  procedures  are  followed,  whether  or  not  command  supervisors  are 
physically  present. 

13.  There  is  redundancy,  multiple  coverage  or  adequate  safety  backups  for  high-risk 
operations. 

14.  Personnel  in  the  command  conduct  continuous  proficiency  training. 

15.  Training  incorporates  safety  guidelines  and  safeguards. 

16.  Leadership  has  vision  and  understands  how  to  create  a  positive  climate. 

17.  The  prevailing  values,  beliefs,  attitudes,  and  norms  (culture)  promote  safe 
behavior/discourages  unsafe  behavior. 

18.  Leadership  sets  example  for  written  policy,  standards,  proper  procedures,  and 
acceptable  norms  of  behavior. 

19.  The  command  clearly  establishes  responsibility  and  accountability,  for  safe  flight 
operations  at  all  levels. 

20.  Command  leaders  have  an  accurate  perception  of  the  motivation,  morale,  and  job 
satisfaction  level  of  their  people. 

21.  Command  leadership  reacts  well,  and  readily  adapts  to  unexpected  changes. 

PROPOSED  COMPONENT  Is  PROCESS  AUDITING 

1 .  My  command  adequately  reviews  and  updates  safety  practices. 

2.  The  command  has  a  dedicated  program  that  targets  individual  training  deficiencies. 

3.  My  command  monitors  maintainer  qualifications. 

4.  Support  equipment  licensing  is  monitored  in  this  command. 

5.  Tool  control  is  taken  seriously  at  my  command. 

6.  CDIs/QARs  routinely  monitor  maintenance  evolutions. 

7.  My  command  uses  safety  staff  to  manage  personnel  at  risk. 

8.  The  command  uses  medical  staff  to  manage  occupational  hazards  and  personnel  at 
risk. 
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PROPOSED  COMPONENT  2:  REWARD  SYSTEM  AND  SAFETY  CULTURE 


1.  My  command  recognizes  individual  safety  achievement  through  rewards  and 
incentives. 

2.  Unprofessional  behavior  is  not  tolerated  in  the  maintenance  department. 

3.  Supervisors  encourage  reporting  safety  concerns  without  fear  of  retribution. 

4.  Supervisors  discourage  violations  of  SOPs,  or  NAMP  guidelines. 

5.  My  MO/MCPO  understands  if  I  feel  uncomfortable  performing  maintenance  duties 
due  to  personal  issues. 

6.  Violations  of  SOP,  NAMP  guidelines,  or  other  procedures  are  common  in  my 
command. 

7.  Peer  influence  discourages  violations  of  SOP,  NAMP  guidelines  or  other  procedures. 

8.  Personnel  are  uncomfortable  telling  supervisors  about  personal  problems  including 
illness. 

9.  Individuals  feel  free  to  report  safety  violations,  unsafe  performance,  or  other  unsafe 
behavior. 

10.  Our  command  climate  promotes  safe  maintenance  and  flight  operations. 

PROPOSED  COMPONENT  3:  QUALITY 

1 .  My  command  has  established  standards  and  maintains  quality  control. 

2.  CDIs/QARs  are  sought  after  positions  in  my  command. 

3.  Inspectors  perform  all  required  actions  before  sign  off. 

4.  To  meet  operational  commitments,  supervisors  allow  “cutting  comers.” 

5.  Maintainer  staffing  is  sufficient  from  shift  to  shift. 

6.  Proper  tools  and  equipment  are  available,  servicable  and  used. 

7.  Required  publications  are  available,  current  and  used. 

8.  Maintenance  gripes  are  either  corrected  or  addressed  prior  to  flight. 

9.  My  command  has  a  reputation  for  quality  maintenance. 

10.  The  QA  division  is  respected  in  my  command. 

11.  Signing  off  PQS/JQRs/PARs  is  taken  seriously  and  not  gun  decked. 

12.  Maintenance  quality  on  detachments  is  the  same  as  that  in  homeport. 

PROPOSED  COMPONENT  4:  RISK  MANAGEMENT 

1.  My  command  temporarily  restricts  maintainers  who  are  having  personal  problems. 

2.  Based  upon  my  command’s  current  manning  and  assets,  it  is  over-committed. 

3.  Supervisors  manage  hazards  associated  with  maintenance  and  flight  line  operations. 

4.  Supervisors  are  more  concerned  with  mission  completion  than  aircraft  maintenance. 

5.  My  division  CPO  is  aware  of  individual  daily  workload  requirements. 

6.  Unsafe  conditions  are  recognized  and  addressed  by  M/C,  Q/A,  or  W/C  supervisors. 

7.  Personnel  turnover  negatively  affects  my  command’s  ability  to  operate  safely. 

8.  Day  and  night  check  have  equal  workload  and  are  equally  stressful/fatiguing. 

9.  Iam  provided  adequate  resources  (time,  personnel,  and  equipment )  to  accomplish  my 
job. 
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10.  Safety  decisions  are  made  at  the  proper  command  levels. 

1 1 .  Safety  is  part  of  maintenance  planning,  and  additional  training/support  is  provided  as 
needed. 

12.  Maintainers  are  never  purposely  put  in  an  unsafe  situation  to  meet  the  flight  schedule. 

PROPOSED  COMPONENT  5:  COMMAND  AND  CONTROL 

1.  My  command  ensures  all  maintainers  are  responsible  and  accountable  for  safe 
maintenance. 

2.  My  command  ensures  the  uniform  enforcement  of  SOPs  among  unit  maintenance 
personnel. 

3.  Supervisors  communicate  command  safety  goals,  programs,  and  procedures. 

4.  Supervisors  are  actively  involved  in  the  safety  program  and  management  of  safety 
matters. 

5.  Supervisors  set  the  example  for  compliance  to  established  maintenance  standards. 

6.  Supervisors  are  responsive  to  unexpected  changes  and  anticipate  potential  hazards. 

7.  W/C  supervisors  are  respected  by  the  maintenance  chief/officer. 

8.  All  maintenance  evolutions  are  properly  supervised  by  qualified  personnel. 

9.  Maintenance  control  is  effective  in  managing  all  maintenance  activities. 

10.  Multiple  job  assignments  and  collateral  duties  adversely  affect  maintenance. 

11.  In  my  command,  we  believe  safety  is  an  integral  part  of  all  maintenance  and  flight 
line  operations. 

12.  Safety  education  and  training  in  my  command  are  comprehensive  and  effective. 

13.  The  safety  department  is  respected  by  supervisors  and  maintainers. 

14.  Maintenance  Safety  Petty  Officer  is  a  sought  after  billet  in  my  command. 

PROPOSED  COMPONENT  6:  COMMUNICATION  /  RELATIONSHIPS 

1 .  My  command  has  a  problem  with  passdown  between  shifts. 

2.  Within  my  unit,  good  communication  flow  exists  up  and  down  the  chain  of  command. 

3.  Coordination  is  conducted  between  the  M/C,  W/C  and  QA  prior  to  incorporation  of 
TDs. 

4.  Work  center  supervisors,  division  CPOs  and  M/C  work  well  together. 

5.  Aircraft  moves  are  briefed  and  detailed  personnel  are  qualified. 

6.  Maintainers  are  briefed  on  potential  hazards  associated  with  maintenance  activities. 

7.  My  supervisor  shields  me  from  outside  pressures,  which  may  affect  my  work. 

8.  QARs  are  never  pressured  by  the  maintenance  supervisors  to  sign  off  a  gripe. 

9.  Maintenance  Control  never  troubleshoots  aircraft  discrepancies. 

10.  QARs  are  viewed  as  helpful,  and  QA  is  not  “feared”  in  my  command. 

1 1. 1  feel  I  get  all  information  (internal  and  external)  required  to  perform  my  job  safely. 
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APPENDIX  B.  CANDIDATE  MAINTENANCE  QUESTIONS 

1 .  My  command  has  a  reputation  for  high  quality  maintenance. 

2.  My  command  is  genuinely  concerned  about  safety. 

3.  Command  leadership  is  successful  in  communicating  its  safety  goals  to  unit 
personnel. 

4.  My  command  provided  a  positive  climate  that  promotes  safe  maintenance  and  flight 
operations. 

5.  Command  leaderships  is  actively  involved  in  the  safety  program  and  management  of 
safety  matter. 

6.  Command  leadership  sets  the  example  for  compliance  to  established  standards. 

7.  My  command  conducts  adequate  reviews  and  updates  of  safety  practices. 

8.  My  command  ensures  that  all  unit  members  are  responsible  and  accountable  for  safe 
maintenance  and  flight  operations. 

9.  Command  leadership  permits  “cutting  comers”  to  get  the  job  done. 

10.  Safety  decisions  are  made  at  the  proper  levels  by  most  qualified  people  in  my 
command. 

1 1 .  Command  leadership  encourages  reporting  safety  discrepancies  without  fear  of 
negative  repercussions. 

12.  Individuals  in  my  command  are  willing  to  report  information  regarding  safety 
violations,  marginal  performance,  or  other  unsafe  behavior. 

13.  My  command  has  a  defined  process  to  set  training  goals  and  to  review  performance. 

14.  Loss  of  experienced  personnel  has  negatively  affected  my  command’s  ability  to 
operate  safely. 

15.  Command  leadership  willingly  assists  in  providing  advice  concerning  safety  matters. 

16.  My  command  has  established  quality  standards  and  strives  to  maintain  quality  control. 

17.  Command  leadership  considers  safety  issues  during  the  formation  and  execution  of 
operational  and  training  plans. 

18.  Command  leadership  has  a  clear  picture  of  the  risks  associated  with  its  maintenance 
and  flight  line  operations. 

19.  My  command  takes  the  time  to  identify  and  assess  risks  associated  with  its 
maintenance  and  flight  line  operations. 

20.  My  command  does  a  good  job  managing  risks  associated  with  its  maintenance  and 
flight  line  operations. 

21.  My  command  closely  monitors  proficiency  and  currency  standards  to  ensure 
maintenance  personnel  are  qualified  to  perform  their  assigned  tasks. 

22.  My  command  provides  adequate  safety  backups  to  catch  possible  human  error  during 
hi-risk  tasks. 

23.  Command  leadership  reacts  well  to  unexpected  changes  to  its  plans. 

24.  In  my  command,  we  believe  safety  is  an  integral  part  of  al  maintenance  and  flight  line 
operations. 

25.  My  command  has  increased  the  chances  of  a  mishap  due  to  inadequate  or  incorrect 
risk  assessment. 
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26.  My  command  does  not  hesitate  to  temporarily  restrict  from  maintenance  tasks 
individuals  who  are  under  high  personal  stress. 

27. 1  am  adequately  trained  to  safely  conduct  all  of  my  assigned  tasks. 

28. 1  am  provided  adequate  resources  (time,  staffing,  budget,  and  equipment)  to 
accomplish  my  job. 

29.  No  equivalent. 

30.  My  collateral  duties  adversely  affect  my  ability  to  accomplish  my  maintenance  tasks. 

31.  Morale  and  motivation  in  my  command  are  high. 

32.  My  command  ensures  the  uniform  enforcement  of  operating  standards  among  unit 
personnel. 

33. 1  have  adequate  time  to  prepare  for  my  assigned  tasks. 

34.  Command  leadership  is  effective  at  discouraging  violations  of  operating  procedures, 
NAMP  guidelines  or  general  maintenance  discipline. 

35.  In  my  command,  peer  influence  is  effective  at  discouraging  violations  of  operating 
procedures,  NAMP  guidelines,  or  general  maintenance  discipline. 

36.  My  command’s  SOP  is  effective  at  promoting  safe  maintenance  and  flight  line 
operations. 

37. 1  am  very  familiar  with  the  policies  and  regulations  contained  in  the  OPNAVINST 
4790.2  series. 

38.  Rest  standards  are  enforced  in  my  command. 

39.  Based  upon  my  command’s  personnel  and  other  assets,  the  command  is  over 
committed. 

40.  In  my  command,  PC/CDI/QAR  tests  and  monitors  are  conducted  as  intended,  to 
candidly  assess  maintainer  qualifications. 

41.  Strict  enforcement  of  NAMP  standards  is  upheld  in  my  command. 

42.  In  my  command,  anyone  who  intentionally  violates  NAMP/NAMSOP  is  swiftly 
corrected. 

43.  In  my  command,  violations  of  operating  procedures,  NAMP  regulations,  or  general 
maintenance  or  flight  line  operation  discipline  are  rare. 

44.  Within  my  command,  good  communication  flow  exists  up  and  down  the  chain  of 
command. 

45.  My  command  has  good  two-way  communication  with  external  commands. 

46.  Procedures  in  my  command  are  adequate  to  effectively  conduct  Human  Factors 
Councils  or  Boards. 

47 .  Human  Factors  councils  have  been  successful  in  identifying  maintenance  members 
who  pose  a  risk  to  safety. 

48.  Human  Factors  Boards  have  been  successful  in  managing  the  high-risk  maintainer. 

49.  My  command  makes  effective  use  of  the  flight  surgeon  to  help  identify  and  manage 
high-risk  personnel. 

50.  My  command  recognizes  an  individual’s  safety  achievements  through  rewards  and 
incentives. 

51.  Safety  education  and  training  are  adequate  in  my  command. 

52.  The  safety  department  is  a  well-respected  element  of  my  command. 

53.  The  Maintenance  Safety  Petty  Officer  position  is  a  sought  after  billet  in  my  command. 
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54.  My  commands  safety  department  keeps  me  well  informed  regarding  important  safety 
information. 

55.  My  command  has  received  adequate  guidance,  information,  and  training  regarding 
Maintenance  Risk  Management. 

56.  My  command  has  begun  to  implement  MRM  processes  into  decision-making  at  all 
levels. 

57.  The  QA  division  is  well  respected  in  my  command. 

58.  CDI  is  a  sought  after  position  in  my  command. 

59.  QAR  is  a  sought  after  position  in  my  command. 

60.  The  EAWS  program  is  considered  to  have  integrity  in  my  command. 

61.  A  coordinated  brief  is  conducted  between  M/C,  the  W/C  and  QA  prior  to 
incorporating  the  AFB/AFC’s,  etc. 

62.  CDI/QARs  perform  routine  monitors  of  maintenance  evolutions. 

63.  Requirements  for  PPE  use  are  strictly  enforced. 

64.  The  qualifying  of  CDI/QAR’s  is  considered  to  be  a  fair  process. 

65.  Decisions  on  who  should  be  CDI/QAR  are  made  fairly  and  without  favoritism. 

66.  Support  equipment  licensing  is  closely  monitored. 

67.  Requirements  for  having  SE  licenses  are  taken  seriously. 

68.  The  “signing-off’  of  PQS/JQRs/PARs,  etc.  is  taken  seriously  and  managed  with 
integrity,  (not  gun  decked) 

69.  Maintenance  Control  provides  for  adequate  time  to  complete  the  job  when  assigning 
tasks. 

70.  Performing  incomplete  daily  inspections  is  not  tolerated  in  my  command. 

71.  Inspectors  never  sign  off  the  “inspected  by”  block  without  physically  performing  the 
required  inspection. 

72.  Work  center  supervisors  work  well  with  M/C. 

73.  W/C  supervisors  are  respected  by  maintenance  management  in  my  command. 

74.  W/C  supervisors  are  respected  by  the  “troops”  in  my  command. 

75.  W/C  supervisors  are  always  asked  their  opinion  by  M/C. 

76.  QA  is  a  “feared”  organization  in  my  command. 

77.  All  participants  involved  in  an  aircraft  move  are  briefed  prior  to  commencing  the 
move. 

78.  All  maintenance  evolutions  are  well  supervised. 

79.  Maintenance  evolutions  are  OVER  supervised. 

80.  Anyone  in  my  command  can  stop  a  maintenance  evolution  if  they  feel  that  an  unsafe 
condition  exists. 

81.  All  pilots  strictly  follow  plane  captain  directions/signals. 

82.  Work  center  shift  passdowns  are  conducted  face  to  face. 

83.  Daily  maintenance  meetings  encourage  feedback  from  work  centers. 

84.  All  work  centers  utilize  a  written  passdown  log. 

85.  Division  CPOs  ensure  100%  participation  at  monthly  maintenance  safety  meetings. 

86.  Workers  with  “bad  attitudes”  are  not  permitted  to  work  on  or  around  aircraft  or  other 
equipment. 

87.  Utilizing  someone’s  NALCOMIS  password  is  strictly  prohibited. 

88.  Maintenance  Control  is  ALWAYS  in  control  of  maintenance. 
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89. 1  have  confidence  that  all  personnel  are  qualified  to  perform  their  assigned  tasks. 

90.  “Down”  gripes  are  taken  seriously  by  QA. 

91.  “Down”  gripes  are  taken  seriously  by  M/C. 

92.  Repeat  gripes  are  monitored  by  QA  and  regarded  as  serious. 

93.  Publications  are  always  used  when  performing  maintenance. 

94.  Maintenance  Control  never  troubleshoots  from  behind  the  M/C  counter. 

95.  Plane  captains  always  carry  daily  decks  when  performing  daily  inpsections. 

96.  A799s  are  taken  seriously. 

97.  Maintenance  training  is  conducted  regularly  and  taken  seriously  by  maintenance 
management. 

98.  CPO’s  are  regarded  as  the  technical  experts  and  get  involved  in  troubleshooting  tough 
gripes. 

99.  CPO’s  over  supervise  maintenance  tasks. 

100.  Tool  Control  is  taken  seriously  at  my  command. 

101 .  The  maintenance  control  chief  is  respected. 

102.  My  CO  is  proud  of  the  maintenance  department. 

103.  Squadron  pilots  feel  comfortable  flying  the  aircraft  in  this  command. 

104.  Our  maintenance  department  is  considered  to  be  the  best  in  the  wing. 

105.  Short  cuts  are  tolerated  when  the  flight  schedule  dictates. 

106.  The  maintenance  department  works  for  the  operations  department. 

107.  The  goals  of  the  maintenance  department  are  clearly  defined. 

108.  Command  leadership  is  more  concerned  with  mission  completion  that  aircraft 
material  conditions. 

109.  PPE  is  available  for  use  when  needed. 

110.  Most  maintenance  is  performed  by  NX. 

111.  NX  maintainers  are  considered  to  be  proficient. 

1 12.  NX  maintainers  are  considered  to  be  as  professional  as  DX. 

113.  “Khaki  presence”  is  evident  at  this  command. 

1 14.  NX  is  well  represented  with  CPO’s. 

115.  Shortcuts  are  tolerated  more  at  night  than  DX. 

1 1 6.  Working  nights  is  more  risky  than  working  days. 

1 17.  Maintainers  would  never  be  put  in  an  unsafe  situation  to  make  the  flight  schedule. 

118.  Maintenance  performed  on  detachments  is  a  lesser  quality  than  that  at  homeport. 

1 19.  W/C  supervisors  are  allowed  to  run  their  shops  without  interference  from  M/C. 

120.  W/C  supervisors  feel  pressure  from  M/C. 

121.  Maintenance  is  only  performed  “by  the  book.” 

122.  Work  priorities  are  clearly  set  at  the  shift  change  maintenance  meetings. 

123.  Use  of  overdue  calibration  equipment  is  not  tolerated. 

124.  Maintenance  personnel  are  often  over  worked. 

125.  Personnel  are  allowed  to  work  on  aircraft  or  equipment  when  “hung-over.” 

126.  Personnel  are  not  comfortable  telling  supervisors  that  they  have  personal 
problems. 

127.  NX  works  longer  shifts  than  DX. 

128.  Personnel  are  assigned  more  than  one  task  at  a  time  making  it  difficult  to 
concentrate  on  one  job. 
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129.  My  maintenance  officer/MCPO  would  understand  if  I  did  not  feel  comfortable 
working  on  an  aircraft  or  equipment  because  I  was  having  a  family  problem. 

130.  Publications  are  available,  understandable,  and  used. 

131.  Iam  comfortable  admitting  to  my  supervisor  that  I  have  made  a  mistake. 

132.  Supervisors  tolerate  “short-cuts.” 

133.  Most  maintainers  would  rather  cover  up  a  mistake  than  admit  to  it. 

134.  Admitting  to  having  an  illness  such  as  a  cold  is  considered  trying  to  “weasel  out 
of  work.” 

135.  My  work  center  is  undermanned. 

136.  My  division  CPO  is  aware  of  my  daily  workload. 

137.  M/C  would  never  put  me  in  work  on  a  job  that  could  put  me  in  an  unsafe 
situation. 

138.  Iam  never  rushed  to  complete  a  task. 

139.  Much  of  the  pressure  from  M/C  to  complete  a  job  is  perceived  and  not  real. 

140.  My  supervisor  is  responsible  for  informing  M/C  if  I  feel  they  are 
pressuring/rushing  me. 

141.  All  known  problems  are  addressed  or  corrected  prior  to  an  aircraft  being  released 
“safe  for  flight.” 

142.  I  am  considered  to  be  an  important  part  of  our  maintenance  department. 

143.  I  feel  comfortable  reporting  someone  who  I  consider  puts  me  in  an  unsafe 
situation. 

144.  Unprofessional  behavior  is  not  tolerated  in  the  maintenance  department. 

145.  Maintenance  management  encourages  me  to  perform  maintenance  “by  the  book.” 

146.  Disciplinary  problems  are  not  considered  by  maintenance  management  to  affect  a 
person’s  ability  to  perform  his/her  job. 

147.  If  an  unsafe  condition  existed,  it  would  be  recognized  by  M/C,  Q/A,  or  the  W/C 
supervisor  before  it  became  a  problem. 

148.  All  personnel  in  my  department  are  physically  capable  of  doing  there  jobs. 

149.  Intentional  violations  of  proper  procedures  are  dealt  with  swiftly. 

1 50.  QA  is  looking  out  for  my  safety. 

151.  I  am  briefed  on  al  possible  hazards  associated  with  performing  a  job  prior  to 
starting  that  job. 

152.  QA  is  considered  to  be  the  technical  experts. 

153.  QA  is  never  pressured  by  maintenance  leadership  to  sign  off  a  gripe. 

154.  My  job  is  as  important  as  the  aircrews. 

155.  My  job  (role)  is  considered  to  be  important  in  achieving  the  overall  command 
goal. 
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APPENDIX  C.  PROTOTYPE  MCAS 


Purpose:  The  purpose  of  this  survey  is  to  try  and  gain  valuable  insight  into  the  maintenance  community’s 
perception  concerning  aviation  mishaps  within  the  Navy  and  Marine  Corps.  Your  participation  and  answers 
will  be  used  as  a  guide  in  the  Navy’s  on-going  efforts  to  lower  the  aviation  mishap  rate. 

The  first  fifteen  questions,  part  I,  regard  biographical  data;  information  particular  to  yourself.  This 
information  will  aid  in  the  analysis  of  your  responses.  NO  attempts  will  be  made  to  identify  individual 
respondents  or  their  organizations. 

Part  II  has  67  questions  pertaining  to  the  maintenance  community.  Please  respond  to  the  questions  with 
the  answer  that  most  correctly  reflects  your  honest  opinion.  Using  a  #2  pencil,  completely  darken  each 
response. 

Thank  you  in  advance  for  your  participation! 

PLEASE  RESPOND  TO  EACH  ITEM. 


1. 

Your 

rank?  (  ) 

E- 

-1  -  E 

-3  ( 

) E-4  -  E-6  ( 

n 

w 

o 

E-7 

+ 

< 

) Officer 

2. 

Your 

community? 

VFA  (  ) 

HS 

(  ) 

VMFA  (  ) 

VF 

( 

) 

HSL  (  ) 

VMA  (  ) 

HC 

(  ) 

VP  (  ) 

HCS 

( 

) 

VX  (  ) 

VR  (  ) 

VQ 

(  ) 

VAQ  (  ) 

VAW 

( 

) 

3  . 

Your 

designator? 

( 

LDO, 

152X, 

etc  )  ? 

_  / 

NEC 

4. 

Are 

you  currently 

a  department  head? 

(  ) 

Yes 

(  )  No 

5. 

Your 

service? 

( 

)  USN 

( 

)  USNR  TAR 

(  ) 

SELRES 

( 

)  Other 

6. 

Your 

shift?  (  ) 

DX  ( 

)  NX 

(  )  MidX 

<  ) 

Other, 

specifv 

7.  Total  years  of  service?  _ 

8.  Total  years  of  Aviation  Maintenance  experience?  _ 

9.  A-School  graduate?  (  )  Yes  (  )  No  (  )  N/A 

10.  Education  level :  (  )  GED  (  )  High  School  (  )  College,  #  of  years _ 

11.  Unit  home  location?  (  ) East  coast  (  )West  Coast  (  ) Other _ 

12.  Your  rating?  (  )  AD/ AM  (  )  AE/AT  (  )  PR/AME  (  )AO  (  )  Other  _ 

13.  Your  age?  {  )  17-20  (  )  21-25  (  )  25-30  (  )  30+ 

14.  Current  maintenance  qualifications? 

{  )Safe  for  Flight  (  )QAR 
(  )  CDI  (  ) Supervisor 

(  )SPO  (  ) N  /A 


15.  Duty:  (  )  Shore  (  )  Sea 
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Part  II 


1 .  My  command  adequately  reviews  and  updates 
safety  practices. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

2.  The  command  has  a  dedicated  program  that 
targets  individual  training  deficiencies. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

3.  My  command  monitors  maintainer  qualifications.  (  ) 

(  ) 

(  ) 

(  ) 

(  ) 

4.  Support  equipment  licensing  is  monitored 
in  this  command. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

5.  Tool  Control  is  taken  seriously  at  my  command. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

6.  My  command  recognizes  individual  safety 
achievement  through  rewards  and  incentives. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

7.  Unprofessional  behavior  is  not  tolerated  in  the 
maintenance  department. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

8.  My  command  has  a  problem  with  passdown 
between  shifts. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

9.  My  command  follows  established  standards 
and  maintains  quality  control. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

10.  CDIs/QARs  are  sought  after  positions  in  my 
command. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

1 1.  My  command  temporarily  restricts  maintainers 
who  are  having  personal  problems. 

(  ) 

(  ) 

(  ) 

(  )  . 

(  ) 

12.  Based  upon  my  command’s  current  manning 
and  assets,  it  is  over-committed. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

13.  My  command  ensures  all  maintainers  are  responsible 
and  accountable  for  safe  maintenance. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

14.  My  command  ensures  the  uniform  enforcement 
of  SOPs  among  unit  maintenance  personnel. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

15.  Within  my  unit,  good  communication  flow 
exists  up  and  down  the  chain  of  command. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

16.  Coordination  is  conducted  between  M/C  ,  W/C 
and  QA  prior  to  incorporation  of  TDs. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

17.  Inspectors  perform  all  required  actions  before 
sign  off. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

18.  Supervisors  encourage  reporting  safety  concerns 
without  fear  of  retribution. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 
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19.  Supervisors  discourage  violations  of  SOPs,  (  ) 

or  NAMP  guidelines. 

20.  To  meet  operational  commitments,  supervisors  (  ) 

allow  “cutting  comers”. 

21.  Supervisors  manage  the  hazards  associated  with  (  ) 

maintenance  and  flight  line  operations. 

22.  Supervisors  are  more  concerned  with  mission  (  ) 

completion  than  aircraft  maintenance. 

23.  My  division  CPO  is  aware  of  individual  (  ) 

daily  workload  requirements. 

24.  Unsafe  conditions  are  recognized  and  (  ) 

addressed  by  M/C,  Q/A,  or  W/C  supervisors. 

25.  Supervisors  communicate  command  safety  goals,  (  ) 

programs  and  procedures. 

26.  Supervisors  are  actively  involved  in  the  safety  (  ) 

program  and  management  of  safety  matters. 

27.  Supervisors  set  the  example  for  compliance  to  (  ) 

established  maintenance  standards. 

28.  Supervisors  are  responsive  to  unexpected  changes  (  ) 

and  anticipate  potential  hazards. 

29.  W/C  supervisors  are  respected  by  the  maintenance  (  ) 

chief/officer. 

30.  All  maintenance  evolutions  are  properly  supervised  (  ) 
by  qualified  personnel. 

31.  Work  center  supervisors  ,  division  CPOs  and  (  ) 

M/C  work  well  together. 

32.  Aircraft  moves  are  briefed  and  detailed  personnel  (  ) 

are  qualified. 

33.  Maintainers  are  briefed  on  potential  hazards  (  ) 

associated  with  maintenance  activities. 

34.  My  supervisor  shields  me  from  outside  (  ) 

pressures  which  may  affect  my  work. 

35.  QARs  are  never  pressured  by  the  maintenance  (  ) 

supervisors  to  sign  off  a  gripe. 

36.  My  MO/MCPO  understand  if  I  feel  uncomfortable  (  ) 
performing  maintenance  due  to  personal  issues. 

37.  Maintainer  staffing  is  sufficient  from  shift  to  shift.  (  ) 
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(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 


(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 


(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 


(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 


38.  Maintenance  control  is  effective  in  managing 
all  maintenance  activities. 

39.  Multiple  job  assignments  and  collateral  duties 
adversely  affect  maintenance. 

40.  Personnel  turnover  negatively  affects  my 
command’s  ability  to  operate  safely. 

41.  Violations  of  SOP,  NAMP  guidelines,  or  other 
procedures  are  common  in  my  command. 

42.  Proper  tools  and  equipment  are  available, 
serviceable  and  used. 

43.  Maintenance  Control  never  troubleshoots 
aircraft  discrepancies. 

44.  Required  publications  are  available,  current, 
and  used. 

45.  Maintenance  gripes  are  either  corrected  or 
addressed  prior  to  flight. 

46.  CDIs/QARs  routinely  monitor  maintenance 
evolutions. 

47.  My  command  has  a  reputation  for  quality  maintenance. 

48.  The  QA  division  is  respected  in  my  command. 

49.  Signing  off  PQS/JQRs/PARs  is  taken  seriously 
and  not  gun  decked. 

50.  Maintenance  quality  on  detachments  is  the  same  (  ) 
as  that  in  homeport. 

5 1 .  Day  and  Night  Check  have  equal  workload  and 
are  equally  stressful/fatiguing. 

52.  QARs  are  viewed  as  helpful,  and  QA  is 
not  “feared”  in  my  command. 

53.  Peer  influence  discourages  violations  of  SOP, 

NAMP  guidelines  or  other  procedures. 

54.  Personnel  are  uncomfortable  telling  supervisors 
about  personal  problems  including  illness. 

55.  Iam  provided  adequate  resources  (time,  personnel 
and  equipment)  to  accomplish  my  job. 

56.  In  my  command,  we  believe  safety  is  an  integral 
part  of  all  maintenance  and  flight  line  operations. 


(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 
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57.  I  feel  I  get  all  information  (internal  and  external) 
required  to  perform  my  job  safely. 

58.  Individuals  feel  free  to  report  safety  violations, 
unsafe  performance,  or  other  unsafe  behavior. 

59.  My  command  uses  safety  staff  to  manage 
personnel  at  risk. 

60.  Our  command  climate  promotes  safe  maintenance 
and  flight  operations. 

61.  Safety  decisions  are  made  at  the  proper 
command  levels. 

62.  Safety  is  part  of  maintenance  planning,  and 
additional  training/support  is  provided  as  needed. 

63.  Maintainers  are  never  purposely  put  in  an 
unsafe  situation  to  meet  the  flight  schedule. 

64.  Safety  education  and  training  in  my  command  are 
comprehensive  and  effective. 

65.  The  safety  department  is  respected  by 
supervisors  and  maintainers. 

66.  Maintenance  Safety  Petty  Officer  is  a  sought  after 
billet  in  my  command. 

67.  The  command  uses  medical  staff  to  manage 
occupational  hazards  and  personnel  at  risk. 


(  )  (  )  (  )  (  )  (  ) 

(  )  (  )  (  )  (  )  (  ) 

(  )  (  )  (  )  (  )  (  ) 

(  )  (  )  (  )  (  )  (  ) 

(  )  (  )  (  )  (  )  (  ) 

(  )  (  )  (  )  (  )  (  ) 

(  )  (  )  (  )  (  )  (  ) 

(  )  (  )  (  )  (  )  (  ) 

(  )  (  )  (  )  (  )  '  (  ) 

(  )  (  )  (  )  (  )  (  ) 

(  )  (  )  (  )  (  )  (  ) 
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APPENDIX  D.  COMPONENT  SURVEY  DESCRIPTIVE  RESULTS 


Question 

Fleet  Support 
Squadron 

Patrol 

Squadron 

Combat  Support 

Squadron 

Combined 

Squadrons 

Total 

o> 

< 

StdDev 

Total 

£ 

CQ 

StdDev 

Total 

> 

< 

CQ 

StdDev 

Total 

EEEI 

StdDev 

1 

108 

4.45 

0.702 

93 

4.45 

0.634 

67 

4.39 

0.758 

268 

4.44 

0.692 

2 

108 

3.86 

0.942 

93 

3.90 

0.835 

67 

3.72 

1.070 

268 

3.84 

0.940 

3 

107 

4.44 

0.881 

92 

4.29 

0.749 

66 

4.27 

0.755 

265 

4.35 

0.849 

4 

106 

4.39 

0.952 

92 

4.36 

0.764 

67 

4.24 

0.836 

265 

4.34 

0.936 

5 

106 

4.54 

0.864 

93 

4.86 

0.406 

67 

4.61 

0.673 

266 

4.67 

0.803 

46 

105 

4.24 

1.020 

90 

4.18 

0.680 

65 

4.20 

0.775 

260 

4.21 

0.960 

59 

104 

3.92 

1.050 

90 

3.91 

0.802 

67 

3.73 

1.109 

261 

3.87 

1.088 

67 

105 

3.30 

1.050 

91 

3.49 

1.004 

66 

3.08 

1.154 

262 

3.31 

1.119 

MOSE  Process  Auditing  Component 


Question 

Fleet  Support 
Squadron 

Patrol 

Squadron 

Combat  Support 
Squadron 

Combined 

Squadrons 

nsEii 

Avg 

iigETl 

■KEll 

Avg 

ESI 

StdDev 

6 

106 

3.60 

0.976 

93 

3.83 

3.828 

67 

3.90 

1.103 

266 

3.78 

1.120 

7 

105 

3.97 

1.222 

92 

4.14 

0.859 

67 

1.100 

264 

3.99 

1.115 

18 

107 

4.27 

0.960 

93 

4.22 

0.845 

67 

4.01 

1.007 

267 

4.17 

0.968 

19 

105 

3.40 

1.350 

92 

4.11 

0.883 

66 

4.03 

0.960 

263 

3.85 

1.216 

36 

101 

3.38 

1.120 

89 

mtw.m 

0.988 

64 

3.17 

1.121 

254 

3.28 

1.187 

41 

105 

2.45 

1.180 

93 

2.11 

1.098 

66 

1.94 

1.006 

264 

2.16 

1.145 

53 

105 

3.68 

1.090 

92 

3.62 

0.959 

66 

3.53 

1.099 

263 

3.61 

1.107 

54 

105 

2.64 

1.220 

91 

2.49 

1.109 

65 

2.91 

1.169 

261 

2.68 

1.202 

58 

105 

4.23 

0.990 

92 

4.35 

0.702 

66 

4.05 

1.143 

4.21 

1.044 

60 

105 

4.38 

0.810 

92 

4.42 

0.683 

67 

4.28 

0.901 

264 

4.36 

0.914 

MOSE  Reward  System  and  Safety  Culture  Component 


Question 

Fleet  Support 
Squadron 

Patrol 

Squadron 

Combat  Support 
Squadron 

Combined 

Squadrons 

Total 

> 

< 

CQ 

StdDev 

Total 

Avg 

StdDev 

Total 

> 

< 

CQ 

StdDev 

Total 

Avg 

StdDev 

9 

107 

4.25 

0.814 

93 

4.30 

0.672 

66 

4.36 

0.572 

266 

4.31 

0.756 

10 

103 

3.70 

1.027 

92 

3.73 

0.800 

65 

3.75 

0.936 

3.73 

1.048 

17 

106 

4.30 

0.790 

92 

4.22 

0.836 

67 

4.24 

0.854 

265 

4.25 

0.898 

20 

106 

1.99 

1.190 

92 

1.82 

1.016 

66 

2.02 

1.143 

264 

1.94 

1.128 

37 

101 

2.70 

1.160 

91 

3.16 

0.981 

67 

2.79 

1.122 

259 

2.89 

1.186 

42 

105 

3.54 

1.300 

90 

3.89 

0.965 

66 

3.89 

1.111 

261 

3.78 

1.215 

44 

107 

4.21 

0.990 

89 

4.33 

0.750 

66 

4.32 

0.914 

262 

4.28 

0.930 

45 

103 

4.32 

1.030 

91 

4.34 

0.749 

66 

4.35 

0.850 

260 

4.34 

1.063 

47 

106 

4.41 

0.880 

92 

4.51 

0.703 

66 

4.62 

0.576 

264 

4.51 

0.847 

48 

105 

4.03 

1.140 

92 

4.34 

0.745 

66 

4.29 

0.855 

263 

4.22 

1.047 

49 

103 

3.84 

1.230 

91 

4.01 

0.960 

66 

3.88 

1.060 

260 

3.91 

1.212 

50 

104 

3.64 

1.270 

92 

4.14 

0.884 

66 

4.00 

1.137 

262 

3.93 

1.219 

MOSE  Quality  Component 
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Question 

Fleet  Support 
Squadron 

Patrol 

Squadron 

Combat  Support 
Squadron 

Combined 

Squadrons 

Total 

Avg 

StdDev 

Total 

ESfefl 

SHiiwsn 

Total 

Avg 

StdDev 

Total 

StdDev 

11 

103 

3.04 

1.056 

91 

3.19 

0.918 

65 

3.28 

0.875 

259 

3.17 

1.050 

12 

105 

2.93 

1.120 

92 

3.15 

0.983 

66 

3.00 

1.123 

263 

3.03 

1.116 

21 

106 

3.94 

1.000 

92 

3.89 

0.805 

66 

3.98 

0.903 

264 

3.94 

0.969 

22 

107 

2.32 

1.290 

92 

2.03 

0.966 

66 

2.15 

1.056 

265 

2.17 

1.135 

23 

105 

3.80 

1.100 

90 

3.73 

0.981 

63 

3.65 

1.109 

258 

3.73 

1.125 

24 

107 

4.33 

0.760 

92 

4.30 

0.752 

66 

4.21 

0.814 

265 

4.28 

0.813 

40 

105 

2.92 

1.270 

91 

2.93 

0.892 

66 

2.86 

1.201 

262 

2.91 

1.165 

51 

103 

2.60 

1.220 

87 

3.21 

0.904 

66 

2.68 

1.166 

256 

2.83 

1.189 

55 

104 

3.50 

1.150 

88 

3.81 

0.981 

67 

3.51 

1.106 

259 

3.60 

1.167 

61 

106 

4.08 

1.020 

92 

4.24 

0.717 

67 

4.12 

0.930 

265 

4.15 

0.964 

62 

104 

4.15 

0.930 

92 

4.27 

0.772 

67 

4.24 

0.818 

263 

4.22 

0.986 

63 

104 

3.80 

1.190 

92 

4.07 

1.003 

66 

4.12 

0.969 

262 

3.99 

1.174 

MOSE  Risk  Management  Component 


Question 

Fleet  Support 
Squadron 

Patrol 

Squadron 

Combat  Support 
Squadron 

Combined 

Squadrons 

Tot 

o> 

> 

< 

SD 

Tot 

SD 

Tot 

Avg 

SD 

Tot 

SD 

13 

106 

4.35 

0.829 

92 

4.33 

0.730 

67 

4.28 

0.692 

265 

4.32 

0.843 

14 

106 

3.98 

0.976 

92 

4.03 

0.870 

66 

4.12 

0.775 

264 

4.04 

0.954 

25 

106 

4.08 

0.950 

92 

4.12 

0.782 

66 

3.98 

0.920 

264 

4.06 

0.951 

26 

106 

3.98 

1.000 

92 

4.18 

0.662 

66 

4.05 

0.919 

264 

4.07 

0.939 

27 

107 

3.95 

0.960 

91 

4.14 

0.754 

66 

3.97 

0.911 

264 

4.02 

0.913 

28 

106 

3.97 

0.950 

92 

4.01 

0.791 

66 

3.88 

0.851 

264 

3.95 

0.933 

29 

107 

3.79 

1.070 

91 

3.45 

1.213 

66 

3.83 

1.090 

264 

3.69 

1.155 

30 

106 

3.85 

1.090 

92 

4.15 

0.725 

66 

4.08 

0.966 

264 

4.03 

1.011 

38 

105- 

3.92 

1.090 

93 

3.82 

0.988 

66 

3.77 

1.049 

264 

3.84 

1.114 

39 

104 

3.30 

1.200 

92 

3.10 

0.973 

66 

3.36 

1.159 

262 

3.25 

1.175 

56 

106 

4.49 

0.810 

92 

4.57 

0.599 

67 

4.61 

0.602 

265 

4.56 

0.792 

64 

105 

4.07 

0.960 

92 

4.10 

0.757 

67 

4.12 

0.879 

264 

4.09 

0.968 

65 

104 

3.92 

1.100 

92 

4.07 

0.823 

67 

4.01 

1.037 

263 

4.00 

1.100 

66 

102 

3.03 

1.100 

90 

3.21 

0.966 

67 

3.10 

1.032 

259 

3.12 

1.126 

MOSE  Command  and  Control  Component 


Question 

Fleet  Support 
Squadron 

Patrol 

Squadron 

Combat  Support 
Squadron 

Combined 

Squadrons 

Total 

Avg 

StdDev 

Total 

Avg 

StdDev 

Total 

Avg 

StdDev 

Total 

ES39I 

StdDev 

8 

102 

2.78 

0.953 

92 

2.38 

1.137 

67 

2.49 

1.146 

261 

2.55 

1.237 

15 

107 

3.75 

1.222 

93 

3.51 

1.039 

67 

3.22 

1.152 

267 

3.49 

1.175 

16 

103 

3.88 

0.953 

91 

3.65 

0.970 

67 

3.73 

0.827 

261 

3.75 

1.055 

31 

106 

3.78 

1.100 

92 

3.60 

1.070 

66 

3.61 

1.149 

264 

3.66 

1.141 

32 

105 

4.00 

1.050 

92 

4.40 

0.696 

66 

3.91 

1.092 

263 

4.10 

1.060 

33 

104 

3.91 

0.980 

91 

4.07 

0.892 

65 

4.00 

0.952 

260 

3.99 

1.053 

34 

103 

2.91 

1.010 

87 

3.21 

1.013 

64 

2.91 

1.019 

254 

3.01 

1.092 

35 

103 

3.66 

1.230 

91 

3.53 

1.036 

66 

3.45 

1.126 

260 

3.55 

1.228 

43 

103 

2.70 

1.210 

90 

2.41 

1.189 

66 

2.61 

1.175 

259 

2.57 

1.236 

52 

106 

3.89 

1.080 

90 

4.08 

0.768 

66 

3.94 

0.943 

262 

3.97 

1.006 

57 

105 

3.96 

1.050 

91 

4.13 

0.748 

67 

4.12 

1.023 

263 

4.07 

1.035 

MOSE  Communication  /  Relationship  Component 


APPENDIX  E.  SUGGESTED  QUESTIONS  MODIFICATIONS 


9.  My  command  adequately  reviews  and  updates  safety  practices. 

1 .  My  command  follows  established  standards  and  maintains  quality  control. 

13.  My  command  ensures  all  maintainers  are  responsible  and  accountable  for  safe 
maintenance. 

New  question:  My  command  adequately  reviews  and  updates  safety  practices,  follows 
established  standards  and  maintains  quality  control,  while  ensuring  that  all  maintainers 
are  responsible  and  accountable  for  safe  maintenance. 

3.  My  command  monitors  maintainer  qualifications. 

4.  Support  equipment  licensing  is  monitored  in  this  command. 

New  question:  My  command  monitors  maintainer  qualifications  and  support  equipment 
licensing. 

57.  I  feel  I  get  all  information  (internal  and  external)  required  to  perform  my  job  safely. 

58.  Individuals  feel  free  to  report  safety  violations,  unsafe  performance,  or  other  unsafe 
behavior. 

New  question:  I  feel  I  get  all  information  required  to  perform  my  job  safely,  and  feel  free 
to  report  safety  violations,  unsafe  performance  or  other  unsafe  behavior. 

30.  All  maintenance  evolutions  are  properly  supervised  by  qualified  personnel. 

33.  Maintainers  are  briefed  on  potential  hazards  associated  with  maintenance  activitites 

New  Question:  Qualified  personnel  properly  supervise  all  maintenance  evolutions  and 
maintainers  are  briefed  on  the  potential  hazards  associated  with  maintenance  activities. 

42.  Proper  tools  and  equipment  are  available,  serviceable  and  used. 

55.  I  am  provided  adequate  resources  (time,  personnel,  and  equipment)  to  accomplish  my 
job. 

New  question:  Proper  tools  and  equipment  are  available,  serviceable  and  used  an  I  am 
provided  adequate  resources  (time,  personnel)  to  accomplish  my  job. 

49.  Signing  off  PQS/JQRs/PARs  is  taken  seriously  and  not  gun  decked. 

50.  Maintenance  quality  on  detachments  is  the  same  as  that  in  homeport. 

New  Question:  Signing  off  PQS/JQRs/PARs  is  taken  seriously,  not  gun  decked  and 
maintenance  quality  is  as  high  on  detachments  as  it  is  in  homeport. 
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2.  The  command  has  a  dedicated  program  that  targets  individual  training  deficiencies. 
14.  My  command  ensures  the  uniform  enforcement  of  SOPs  among  maintenance 
personnel. 

New  question:  My  command  has  a  dedicated  program  that  targets  individual  training 
deficiencies  and  ensures  the  uniform  enforcement  of  SOPs  among  maintenance 
personnel. 

37.  Maintainer  staffing  is  sufficient  from  shift  to  shift. 

5 1 .  Day  and  night  check  have  equal  workload  and  are  equally  stressful  /  fatiguing. 

New  question:  Maintainer  staffing  is  sufficient,  is  equally  worked  and  is  equally  stressed 
/  fatigued  from  shift  to  shift. 

16.  Coordination  is  conducted  between  M/C,  W/C,  and  QA  prior  to  incorporation  of  TDs. 

38.  Maintenance  control  is  effective  in  managing  all  maintenance  activities. 

New  question:  Maintenance  control  is  effective  in  managing  all  maintenance  activities, 
coordinating  between  M/C,  W/C,  and  QA  prior  to  the  incorporations  of  TDs. 

20.  To  meet  operational  commitments,  supervisors  allow  cutting  comers. 

22.  Supervisors  are  more  concerned  with  mission  completion  than  aircraft  maintenance. 

New  question:  Supervisors  are  more  concerned  with  proper  aircraft  maintenance  than 
mission  completion  and  do  not  allow  cutting  comers  to  meet  operational  commitments. 

64  Safety  education  and  training  in  my  command  are  comprehensive  and  effective. 

65.  The  safety  department  is  respected  by  supervisors  and  maintainers. 

New  question:  Safety  education  and  training  in  my  command  are  comprehensive  and 
effective  and  the  safety  department  is  respected  by  the  supervisors  and  maintainers. 


52.  QARs  are  viewed  as  helpful,  and  QA  is  not  “feared”  in  my  command. 

56.  In  my  command,  we  believe  safety  is  an  integral  part  of  all  maintenance  and  flight 
line  operations. 

New  question:  Safety  is  an  integral  part  of  this  command’s  maintenance  planning/flight 
line  operations,  where  QARs  are  helpful  and  the  QA  division  is  not  “feared”. 

19.  Supervisors  discourage  violations  of  SOPs,  or  NAMP  guidelines. 

53.  Peer  influence  discourages  violations  of  SOP,  NAMP  guidelines  or  other  procedures. 

New  question:  Violations  of  SOP,  NAMP  guidelines  or  other  procedures  are  discouraged 
in  this  command. 
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5.  Tool  control  is  taken  seriously  in  my  command. 

47.  My  command  has  a  reputation  for  quality  maintenance. 

New  question:  My  command  has  a  reputation  for  quality  maintenance  and  tool  control  is 
taken  seriously. 

18.  Supervisors  encourage  reporting  safety  concerns  without  fear  of  retribution. 

21.  Supervisors  manage  the  hazards  associated  with  maintenance  and  flight  line 
operations. 

24.  Unsafe  conditions  are  recognized  and  addressed  by  M/C,  Q/A,  or  W/C  supervisors. 

New  question:  Safety  concerns  or  unsafe  hazards  associated  with  maintenance  and  flight 
line  operations  can  be  reported  without  a  fear  of  retribution  knowing  that  the  W/C,  Q/A, 
or  M/C  supervisors  will  address  and  manage  them  for  proper  corrections. 

11.  My  command  temporarily  restricts  maintainers  who  are  having  personal  problems. 
36.  My  MO/MCPO  understand  if  I  feel  uncomfortable  performing  maintenance  due  to 
personal  issues 

67.  The  command  uses  medical  staff  to  manage  occupational  hazards  and  personnel  at 
risk. 

New  question:  Medical  staff  is  used  to  help  identify,  manage,  and  temporarily  restrict 
personnel  with  personal  issues  and  those  who  pose  a  risk  to  safe  maintenance  in  this 
command. 

10.  CDIs  and  QARs  are  sought  after  positions  in  my  command. 

66.  Maintenance  Safety  Petty  Officer  is  a  sought  after  billet  in  my  command. 

New  question:  QARs/  CDIs  and  Maintenance  Safety  Petty  Officer  are  sought  after 
billets  in  my  command. 

45.  Maintenance  gripes  are  either  corrected  or  addressed  prior  to  flight. 

46.  CDIs/QARs  routinely  monitor  maintenance  evolutions. 

48.  The  QA  division  is  respected  in  my  command. 

New  question:  The  QA  division  is  respected  and  CDIs  /  QARs  routinely  monitor 
maintenance  evolutions  ensuring  that  maintenance  gripes  are  either  corrected  or 
addressed  prior  to  flight. 

Question  8: 

Current:  My  command  has  a  problem  with  passdown  between  shifts. 

New  question:  My  command  does  not  have  a  problem  with  passdown  between  shifts. 
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Question  12: 


Current:  Based  upon  my  command’s  current  manning  and  assets,  it  is  over-committed. 
New  question:  Based  upon  my  command’s  current  manning  and  assets,  it  is  not  over¬ 
committed. 

Question  20 

Current:  To  meet  operational  commitments,  supervisors  allow  “cutting  comers.” 

New  question:  Supervisors  do  not  allow  “cutting  comers”  to  meet  operational 
commitments. 

Question  22 

Current:  Supervisors  are  more  concerned  with  mission  completion  than  aircraft 
maintenance. 

New  question:  Supervisors  are  more  concerned  with  proper  aircraft  maintenance  than 
mission  completion. 

Question  39 

Current:  Multiple  job  assignments  and  collateral  duties  adversely  affect  maintenance. 

New  question:  Multiple  job  assignments  and  collateral  duties  do  not  adversely  affect 
maintenance. 

Question  40 

Current:  Personnel  turnover  negatively  affects  my  command’s  ability  to  operate  safely. 
New  question:  Personnel  turnover  does  not  affect  my  command’s  ability  to  operate 
safely. 


Question  41 

Current:  Violations  of  SOP,  NAMP  guidelines,  or  procedures  are  common  in  my 
command. 

New  question:  Violations  of  SOP,  NAMP  guidelines,  or  procedures  are  not  common  in 
my  command. 


74 


Question  54 


Current:  Personnel  are  uncomfortable  telling  supervisors  about  personal  problems 
including  illness. 

New  question:  Personnel  are  comfortable  telling  supervisors  about  personal  problems 
including  illness. 
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APPENDIX  F.  FINALIZED  MCAS 


Purpose:  The  purpose  of  this  survey  is  to  try  and  gain  valuable  insight  into  the  maintenance  community’s 
perception  concerning  aviation  mishaps  within  the  Navy  and  Marine  Corps.  Your  participation  and  answers 
will  be  used  as  a  guide  in  the  Navy’s  on-going  efforts  to  lower  the  aviation  mishap  rate. 

The  first  fifteen  questions,  part  I,  regard  biographical  data;  information  particular  to  yourself.  This 
information  will  aid  in  the  analysis  of  your  responses.  NO  attempts  will  be  made  to  identify  individual 
respondents  or  their  organizations. 

Part  II  has  35  questions  pertaining  to  the  maintenance  community.  Please  respond  to  the  questions  with 
the  answer  that  most  correctly  reflects  your  honest  opinion.  Using  a  #2  pencil,  completely  darken  each 
response. 

Thank  you  in  advance  for  your  participation ! 

PLEASE  RESPOND  TO  EACH  ITEM. 

1.  Your  rank?  (  )  E-l  -  E-3  (  )E-4  -  E-6  (  ) CPO  E-7  +  (  ) Officer 

2 .  Your  community? 


VFA 

(  ) 

HS  ( 

)  VMFA  ( 

) 

VF 

(  ) 

HSL  ( 

) 

VMA 

<  ) 

HC  ( 

)  VP  ( 

) 

HCS 

(  ) 

VX  ( 

) 

VR 

(  > 

VQ  ( 

)  VAQ  ( 

) 

VAW 

(  ) 

3.  Your  designator?  (  LDO,  152X,  etc  )?  _  /  NEC  _ 

4 .  Are  you  currently  a  department  head?  (  )  Yes  (  )  No 

5.  Your  service?  (  )  USN  (  )  USNR  TAR  (  )  SELRES  (  )  Other 

6.  Your  shift?'  (  )  DX  (  )  NX  {  )  MidX  (  )  Other _ 

7.  Total  years  of  service?  _ 

8.  Total  years  of  Aviation  Maintenance  experience?  _ 

9.  A-School  graduate?  (  )  Yes  (  )  No  (  )  N/A 

11.  Education  level:  (  )  GED  (  )  High  School  (  )  College,  #yrs _ 

12.  Unit  home  location?  {  )East  coast  (  )West  Coast  (  )  Other 

12.  Your  rating?  (  )  AD/AM  (  )  AE/AT  (  )  PR/AME  (  )AO  (  )  Other 

13.  Your  age?  (  )  17-20  (  )  21-25  (  )  25-30  (  )  30+ 

14.  Current  maintenance  qualifications? 

(  )Safe  for  Flight  (  ) QAR 
{  )  CDI  (  ) Supervisor 

{  )SPO  (  ) N  /A 

15.  Duty:  (  )  Shore  (  )  Sea 
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1 .  My  command  has  a  dedicated  program  that 
targets  individual  training  deficiencies  and  ensures 
the  uniform  enforcement  of  SOPs  among  maintenance 
personnel. 

2.  My  command  monitors  maintainer  qualifications 
and  support  equipment  licensing. 

3.  My  command  has  a  reputation  for  quality 
maintenance  and  tool  control  is  taken  seriously. 

4.  Unprofessional  behavior  is  not  tolerated  in  the 
maintenance  department. 

5.  My  command  has  a  problem  with  passdown 
between  shifts. 

6.  My  command  adequately  reviews  and  updates 
safety  practices,  follows  established  standards  and 
maintains  quality  control,  ensuring  that  all  maintainers 
are  responsible  and  accountable  for  safe  maintenance. 

7.  QARs/  CDIs  and  Maintenance  Safety  Petty  Officer  are 
sought  after  billets  in  my  command. 

8  .  Medical  and  safety  staff  are  used  to  help  identify, 

manage,  and  temporarily  restrict  personnel  with  personal 
issues  and  those  who  pose  a  risk  to  safe  maintenance  in 
this  command. 

9.  Based  upon  my  command’s  current  manning 
and  assets,  it  is  not  over-committed. 

10.  Within  my  unit,  good  communication  flow 
exists  up  and  down  the  chain  of  command. 

1 1 .  Maintenance  control  is  effective  in  managing  all 
maintenance  activities,  coordinating  between  M/C, 
W/C,  and  QA  prior  to  the  incorporations  of  TDs. 

12.  Inspectors  perform  all  required  actions  before 
sign  off. 

13.  Safety  concerns  or  unsafe  hazards  associated 
with  maintenance/flight  line  operations  can  be 
reported  without  fear  of  retribution  knowing  that  the 
W/C,  Q/A,  or  M/C  supervisors  will  address  and  manage 
them  for  proper  corrections. 

14.  Violations  of  SOP,  NAMP  guidelines  or  other 
procedures  are  discouraged  in  this  command. 


15.  Supervisors  are  more  concerned  with  proper  aircraft  (  )  (  )  (  )  (  )  (  ) 

maintenance  than  mission  completion  and  do  not 

allow  cutting  comers  to  meet  operational  commitments. 

16.  My  supervisors  are  aware  of  individual  (  )  (  )  (  )  (  )  (  ) 

daily  workload  requirements  and  recognize  safety 

achievements  through  rewards  and  incentives. 

17.  Supervisors  communicate  command  safety  goals,  (  )  (  )  (  )  (  )  (  ) 

programs  and  procedures. 

18.  W/C  supervisors  are  respected  by  the  maintenance  (  )  (  )  (  )  (  )  (  ) 

chief/officer. 

19.  Qualified  personnel  properly  supervise  all  (  )  (  )  (  )  (  )  (  ) 

maintenance  evolutions  and  maintainers  are 

briefed  on  the  potential  hazards  associated  with 
maintenance  activities. 

20.  My  supervisor  shields  me  from  outside  (  )  (  )  (  )  (  )  (  ) 

pressures  which  may  affect  my  work. 

21.  QARs  are  never  pressured  by  the  maintenance  (  )  (  )  (  )  (  )  (  ) 

supervisors  to  sign  off  a  gripe. 

22.  Maintainer  staffing  is  sufficient,  is  equally  worked  and  (  )  (  )  (  )  (  )  (  ) 

is  equally  stressed  /  fatigued  from  shift  to  shift. 

23.  Multiple  job  assignments  and  collateral  duties  (  )  (  )  (  )  (  )  (  ) 

do  not  adversely  affect  maintenance. 

24.  Personnel  turnover  does  not  affect  my  (  )  (  )  (  )  (  )  (  ) 

command’s  ability  to  operate  safely. 

25.  Violations  of  SOP,  NAMP  guidelines,  or  other  (  )  (  )  (  )  .(  )  (  ) 

procedures  are  not  common  in  my  command. 

26.  Proper  tools  and  equipment  are  available,  (  )  (  )  (  )  (  )  (  ) 

serviceable  and  used  and  I  am  provided  adequate 

resources  (time,  personnel)  to  accomplish  my  job. 

27.  Maintenance  Control  never  troubleshoots  (  )  (  )  (  )  (  )  (  ) 

aircraft  discrepancies. 

28.  Required  publications  are  available,  current,  (  )  (  )  (  )  (  )  (  ) 

and  used. 

29.  The  QA  division  is  respected  and  CDIs  /  QARs  (  )  (  )  (  )  (  )  (  ) 

routinely  monitor  maintenance  evolutions  ensuring 

that  maintenance  gripes  are  either  corrected  or 
addressed  prior  to  flight. 

30.  Signing  off  PQS/JQRs/PARs  is  taken  seriously,  (  )  (  )  (  )  (  )  (  ) 

not  gun  decked  and  maintenance  quality  is  as  high 

on  detachments  as  it  is  in  homeport. 
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31.  Safety  is  an  integral  part  of  this  command’s  (  ) 

maintenance  planning/flight  line  operations,  where 

QARs  are  helpful  and  the  QA  division  is  not  “feared”. 

(  ) 

(  ) 

( > 

(  ) 

32.  Personnel  are  comfortable  telling  supervisors  (  ) 

about  personal  problems  including  illness. 

(  ) 

(  ) 

( > 

(  ) 

33.  I  feel  I  get  all  information  (internal  and  external)  (  ) 

required  to  perform  my  job  safely,  and  feel  free  to 
report  safety  violations,  unsafe  performance  or  other 
unsafe  behavior. 

(  ) 

(  ) 

( > 

(  ) 

34.  Maintainers  are  never  purposely  put  in  an  (  ) 

unsafe  situation  to  meet  the  flight  schedule. 

(  ) 

(  ) 

( > 

(  ) 

35  Safety  education  and  training  in  my  command  are  (  ) 

comprehensive  and  effective  and  the  safety  department 
is  respected  by  the  supervisors  and  maintainers. 

(  ) 

(  ) 

( > 

(  ) 
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